Papers with transfer learning
Investigating label suggestions for opinion mining in German Covid-19 social media (2021.acl-long)
Copied to clipboard
| Challenge: | Existing difficulties in data annotation are due to prolonged data gathering processes or opinion surveys being subject to reactivity. |
| Approach: | They propose to use label suggestions to improve annotation efficiency in german Covid-19 data by providing annotators with pre-recorded annotations. |
| Outcome: | The proposed model improves inter-annotator agreement and annotation quality in a controlled study with social science students. |
Named Entity Recognition Under Domain Shift via Metric Learning for Life Sciences (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing models for named entity recognition fail in scientific domains such as biomedicine and chemistry. |
| Approach: | They propose a model to transfer knowledge from the biomedical domain to the target domain . they use pseudo labeling and contrastive learning to enhance discrimination . |
| Outcome: | The proposed model outperforms baseline models by up to 5% . the proposed model is based on a biomedical domain model and a chemical domain model . |
Evaluating zero-shot transfers and multilingual models for dependency parsing and POS tagging within the low-resource language family Tupían (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing studies on NLP applications for low-resource languages have not been done in this area. |
| Approach: | They propose to replicate the transferability of dependency parsers and POS taggers trained on closely related languages within the low-resource language family Tupan. |
| Outcome: | The proposed models replicate the transferability of dependency parsers and POS taggers trained on closely related languages within the low-resource language family Tupan. |
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (D19-56)
Copied to clipboard
| Challenge: | Statistical conversational systems are complex, timeintensive, expensive, and not easily transferable due to data scarcity. |
| Approach: | They propose a task-oriented dialogue model that operates on text input . they validate it on multi-domain task-orientated dialogues from a multi-word dataset . |
| Outcome: | The proposed model bypasses explicit policy and language generation modules on multi-domain task-oriented dialogues from the MultiWOZ dataset. |
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word2vec-based methods for learning rare or unseen words have been criticized for degrading performance in small corpus settings. |
| Approach: | They propose a la carte embedding method that relies on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. |
| Outcome: | The proposed method is based on a new dataset showing that it can be used when a word is encountered even if only a single usage example is available. |
Revisiting Multi-Domain Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to handle multi-domain machine translation systems are lacking due to the variability of data. |
| Approach: | They propose to use domain adaptation methods to handle situations where a sample of matched sentences is available in training and where only samples of source-side sentences are available. |
| Outcome: | The proposed model is able to handle multiple domains and their expectations with respect to performance. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Practical Semantic Parsing for Spoken Language Understanding (N19-2)
Copied to clipboard
| Challenge: | Existing systems that can handle a user's utterance are unable to handle Q&A or SLU. |
| Approach: | They build a transfer learning framework for executable semantic parsing . they show it is effective for Q&A and for spoken language understanding . |
| Outcome: | The proposed framework is effective for Q&A and Spoken Language Understanding . it can be learned by exploiting data on other domains, the authors show . |
Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have used pre-trained language models for domain adaptation or transfer learning to improve natural language processing performance. |
| Approach: | They propose to fine-tune word sense disambiguation on sequence-pair ranking task and to use existing WordNet examples to augment the model. |
| Outcome: | The proposed model achieves state-of-the-art on the English all-words benchmark datasets. |
Transfer Learning Between Related Tasks Using Expected Label Proportions (D19-1)
Copied to clipboard
| Challenge: | Existing methods of data supervision are limited by labeled training data. |
| Approach: | They propose a method where models are trained based on expected label proportions. |
| Outcome: | The proposed method improves on a sentence-level sentiment predictor and is cumulative with LM-based pretraining. |
Fairseq S2T: Fast Speech-to-Text Modeling with Fairseq (2020.aacl-demo)
Copied to clipboard
| Challenge: | End-to-end sequence-to sequence (S2S) modeling has witnessed rapid growth in speech-totext (ST) tasks. |
| Approach: | They introduce fairseq S2T, a fairsq extension for speech-to-text modeling tasks such as end-to end speech recognition and speech-text translation. |
| Outcome: | The proposed extension provides end-to-end workflows from data pre-processing, model training to offline (online) inference. |
Research on Task Discovery for Transfer Learning in Deep Neural Networks (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing deep neural network based machine learning models suffer from overfitting and are sensitive to noise and examples that are not available in training data. |
| Approach: | They propose to use a novel multi-task learner to implement deep neural network based transfer learning models that can be used to improve generalization. |
| Outcome: | The proposed model performs better on two NLP tasks and is more efficient on other areas of machine learning, including Bioinformatics and Computer Vision. |
AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters (2022.acl-demo)
Copied to clipboard
Tilman Beck, Bela Bohlender, Christina Viehmann, Vincent Hane, Yanik Adamson, Jaber Khuri, Jonas Brossmann, Jonas Pfeiffer, Iryna Gurevych
| Challenge: | AdapterHub Playground is an open-access tool for researchers to use pretrained language models without writing a single line of code. |
| Approach: | They propose a tool which allows researchers to leverage pretrained models without writing a single line of code for a variety of NLP tasks. |
| Outcome: | The proposed model can be used for prediction, training and analysis of textual data without writing a single line of code. |
The Amazing World of Neural Language Generation (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in neural text generation due to advances in deep contextual language modeling and transfer learning. |
| Approach: | They will discuss how and why NLG models succeed/fail at generating coherent text. |
| Outcome: | This paper will discuss how and why these models succeed/fail at generating coherent text, and provide insights on several applications. |
ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus (2021.acl-demo)
Copied to clipboard
| Challenge: | 7000 languages worldwide are spoken, but most research is focused on English . multilinguality is essential for multilingual research, and is a key component of the process. |
| Approach: | They propose a wordaligned parallel corpus that can be browsed using an online tool . they use the word alignment tools SimAlign and BabelNet to find the alignments . |
| Outcome: | The proposed tool can be set up for any parallel corpus and explores its quality and properties. |
Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages (2024.naacl-srw)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a useful component in NLP applications. |
| Approach: | They propose to use annotated named entity corpora to classify a given entity into a category within a textual document. |
| Outcome: | The proposed model achieves an F1 score of 0.80 on an unseen dataset for Indian languages. |
DeezyMatch: A Flexible Deep Learning Approach to Fuzzy String Matching (2020.emnlp-demos)
Copied to clipboard
| Challenge: | DeezyMatch is a free, open-source software library written in Python for fuzzy string matching and candidate ranking. |
| Approach: | They propose to use DeezyMatch to train new classifiers and fine-tune a pretrained model to generate rich vector representations from string inputs. |
| Outcome: | The proposed algorithm can be used to find the best matching candidates in large knowledge bases and query sets. |
ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format (2023.emnlp-demo)
Copied to clipboard
Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gasic, Minlie Huang
| Challenge: | Existing tools for building TOD systems often lack a user-friendly interface . a toolkit with advanced, easily integrable modules is needed to bridge this gap . |
| Approach: | They propose a multifaceted dialogue system toolkit that integrates diverse datasets and models with a streamlined training process and in-depth evaluation tools. |
| Outcome: | The proposed toolkit combines RL and transfer learning to support the rapid development and evaluation of robust dialogue policies. |
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods for transferring knowledge from BERT into a model with large parameters are not efficient due to their large-scale and high computational cost. |
| Approach: | They propose a sentence representation approximating oriented distillation framework that can distill pre-trained BERT into a simple LSTM based model without specifying tasks. |
| Outcome: | The proposed model outperforms other distillation methods and larger models on multiple NLP tasks with efficiency well-improved. |
Cross-Task Knowledge Transfer for Query-Based Text Summarization (D19-58)
Copied to clipboard
| Challenge: | Existing methods for summarization data corpora are limited to extractive and abstractive summarizing. |
| Approach: | They propose to use machine reading comprehension (MRC) and query-based text summarization to produce extractive and abstractive summaries from pre-trained MRC and MT models. |
| Outcome: | The proposed model outperforms existing methods on CNN/Daily Mail and Debatepedia datasets and can be used as a baseline for future systems. |
Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention (2020.coling-industry)
Copied to clipboard
| Challenge: | Existing methods to integrate hypotheses into speech recognition systems are noisy and can cause information loss. |
| Approach: | They propose to integrate hypotheses into multi-task learning and transfer learning to improve performance. |
| Outcome: | The proposed model improves domain and intent classification by 19% and 37% compared to current methods . the proposed model could recover transcription and rewrite the query for a better understanding . |
Transferable Dialogue Systems and User Simulators (2021.acl-long)
Copied to clipboard
| Challenge: | a lack of training data is limiting the development of dialogue systems . we develop a framework for creating dialogue data through self-play between agents . |
| Approach: | They propose a framework that can incorporate new dialogue scenarios through self-play between two agents. |
| Outcome: | The proposed framework is highly effective in bootstrapping the performance of two agents in transfer learning. |
Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settings (D19-61)
Copied to clipboard
| Challenge: | Recent research points to knowledge distillation as a potential solution for NLU tasks. |
| Approach: | They propose a training approach that distills large finetuned LMs into a small network using unlabeled training examples. |
| Outcome: | The proposed approach outperforms BERT training approaches while using 300 times fewer parameters. |
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models (2020.acl-demos)
Copied to clipboard
Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, Samuel R. Bowman
| Challenge: | jiant is an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. |
| Approach: | They introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. |
| Outcome: | The proposed toolkit reproduces published performance on GLUE and SuperGLUE tasks. |
Transfer Learning in Biomedical Named Entity Recognition: An Evaluation of BERT in the PharmaCoNER task (D19-57)
Copied to clipboard
| Challenge: | Existing methods for natural language processing are labor-intensive and skill-dependent . Currently, most biomedical natural language tasks focus on English documents . |
| Approach: | They introduce a BERT benchmark to facilitate the research of PharmaCoNER task . they evaluate two baselines based on Multilingual BERT and BioBERT on the corpus . |
| Outcome: | The proposed task is based on multilingual BERT and BioBERT on the PharmaCoNER corpus. |
Benchmarks and models for entity-oriented polarity detection (N18-3)
Copied to clipboard
| Challenge: | a dataset of 17,000 manually labeled documents is large for determining entity-oriented polarity in business news. |
| Approach: | They propose a convolutional neural network-based approach to classify entity-oriented polarity in business news. |
| Outcome: | The proposed model is based on convolutional neural networks and is small on the scale of existing models. |
Parallel sentences mining with transfer learning in an unsupervised setting (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to mine parallel sentences in low-resource environments are not suitable for many low-level language pairs. |
| Approach: | They propose an approach based on transfer learning to mine parallel sentences in an unsupervised setting using bilingual corpora of low-resource language pairs. |
| Outcome: | The proposed model improves the performance of mined parallel sentences at two real-world low-resource language pairs compared with previous methods. |
Multilingual Neural Semantic Parsing for Low-Resourced Languages (2021.starsem-1)
Copied to clipboard
| Challenge: | a large amount of training data is needed to understand multilingual semantic parsing models. |
| Approach: | They propose to use machine translation to bootstrap multilingual training data from English data. |
| Outcome: | The proposed model outperforms existing models on human-written sentences and the state-of-the-art models on the public NLMaps dataset. |
Script Parsing with Hierarchical Sequence Modelling (2021.starsem-1)
Copied to clipboard
| Challenge: | Script knowledge is a category of commonsense knowledge that describes how people conduct everyday activities sequentially. |
| Approach: | They propose a hierarchical sequence model and transfer learning to do script parsing with a sequence model that accurately tags script participants. |
| Outcome: | The proposed model improves state of the art of event parsing by over 16 points F-score and, for the first time, accurately tags script participants. |
Aligning Generative Language Models with Human Values (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for learning human values do not consider contextual and abstract nature of human values. |
| Approach: | They propose a reinforcement learning based method that embeds human values judgements into each step of language generation. |
| Outcome: | The proposed method improves on human values judgements and shows higher alignment performance. |
Statistical Deficiency for Task Inclusion Estimation (2025.acl-long)
Copied to clipboard
Loïc Fosse, Frederic Bechet, Benoit Favre, Géraldine Damnati, Gwénolé Lecorvé, Maxime Darrin, Philippe Formont, Pablo Piantanida
| Challenge: | Tasks are central in machine learning, as they are the most natural objects to assess the capabilities of current models. |
| Approach: | They propose a theoretically grounded setup to define the notion of task and compute the inclusion between two tasks from a statistical deficiency point of view. |
| Outcome: | The proposed model estimates the degree of inclusion between tasks on synthetic data and reconstructs the classic NLP pipeline. |
Fast and Scalable Expansion of Natural Language Understanding Functionality for Intelligent Agents (N18-3)
Copied to clipboard
| Challenge: | a recent paper describes efficient deep neural network architectures for expanding natural language capabilities of virtual agents. |
| Approach: | They propose deep neural network architectures that maximize re-use available resources . they use data from Amazon Alexa to accelerate expansion of new natural language domains . |
| Outcome: | The proposed methods increase accuracy in low resource settings and enable rapid development with less data. |
A Named Entity Recognition Shootout for German (P18-2)
Copied to clipboard
| Challenge: | Named entity recognition and classification (NER) is a central component in many natural language processing pipelines. |
| Approach: | They propose to build a model for German named entity recognition that performs at the state of the art for both contemporary and historical texts. |
| Outcome: | The proposed model outperforms the CRF and BiLSTM on large and small datasets. |
Multi-lingual neural title generation for e-Commerce browse pages (N18-3)
Copied to clipboard
| Challenge: | e-Commerce websites are automatically generating millions of browse pages . manual creation of titles is infeasible due to the huge number of browse page types . |
| Approach: | They propose to use sequence-to-sequence models to generate titles for languages . they train the models on multi-lingual data, thereby creating one joint model . |
| Outcome: | The proposed model can generate titles in three different languages, with a focus on low-resource French. |
How many words does it take to understand a low-resource language? (2025.naacl-srw)
Copied to clipboard
| Challenge: | We evaluated the documentation needed to create a sentence embedding space using widely spoken languages. |
| Approach: | They propose to use widely spoken languages as a proxy for low-resource languages to evaluate the documentation needed to create a sentence embedding space. |
| Outcome: | The proposed language model can be used to improve the performance of sentences embedded in low-resource languages. |
Exploring Multilingual Syntactic Sentence Representations (D19-55)
Copied to clipboard
| Challenge: | Recent studies on language models that learn syntactic information focus on learning the semantic structures of language. |
| Approach: | They propose to use a multilingual parallel corpus augmented by universal part-of-speech tags to learn syntactic sentence embeddings. |
| Outcome: | The proposed method performs better than state-of-the-art language models in low-resource languages. |
YANMTT: Yet Another Neural Machine Translation Toolkit (2023.acl-demo)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is an end-to-end approach that provides stateof-the-art results for a variety of language pairs. |
| Approach: | They propose to build an open-source neural machine translation toolkit on top of HuggingFace's Transformers library and use it for pre-training and fine-tuning sequence-to-sequence models. |
| Outcome: | The proposed toolkit is built on top of the HuggingFace Transformers library and provides advanced features such as document/multi-source NMT, simultaneous NMT and mixtures-of-experts. |
T2NER: Transformers based Transfer Learning Framework for Named Entity Recognition (2021.eacl-demos)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is an important task in information extraction due to large variations in entity names and flexibility in how entities are mentioned. |
| Approach: | They propose a Transformers based Transfer Learning framework for Named Entity Recognition (T2NER) that integrates transformer models with the state-of-the-art in NLP and provides a unified platform for transfer learning. |
| Outcome: | The proposed framework bridges the gap between the state-of-the-art in transformer models and the state of the art in NER with deep transformer models. |
CMTA: COVID-19 Misinformation Multilingual Analysis on Twitter (2021.acl-srw)
Copied to clipboard
| Challenge: | myths, sensationalism, rumours and misinformation, generated intentionally or unintentionally, spread rapidly through social networks during the COVID-19 pandemic . evaluation of tweets for recognizing misinformation can create beneficial understanding to review the top quality and also the readability of online information concerning the COV-19. |
| Approach: | They propose a multilingual COVID-19 related tweet analysis method that uses a deep learning model for multilingual tweet misinformation detection and classification. |
| Outcome: | The proposed method outperforms monolingual models in the misinformation detection task and shows that it can be used to improve the quality and readability of online information. |
Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages (2026.acl-short)
Copied to clipboard
| Challenge: | Existing studies show that performance across low-resource settings is variable, resulting in a significant barrier for the MT community. |
| Approach: | They propose to use FRED Difficulty Metrics to contextualize reported performance across different language pairs to determine whether breakthroughs reported in other contexts are artifacts of benchmark collection. |
| Outcome: | The proposed metrics explain a significant portion of result variability rather than model capability. |
Universal Sentence Encoder for English (D18-2)
Copied to clipboard
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, Ray Kurzweil
| Challenge: | TensorFlow Hub sentence embedding models have good task transfer performance . model variants allow for trade-offs between accuracy and compute resources . |
| Approach: | They propose easy-to-use TensorFlow Hub sentence embedding models with good task transfer performance. |
| Outcome: | The proposed models outperform models without transfer learning and those that use only word-level transfer on a number of NLP tasks. |
Multilingual Argument Mining: Datasets and Analysis (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Argument mining tasks in non-English languages are dominated by English . we use a pre-trained language model that supports 104 languages to train models . |
| Approach: | They propose a multilingual BERT model to address argument mining tasks in non-English languages . they use English datasets and machine translation to facilitate transfer learning . |
| Outcome: | The proposed model is well suited for classifying the stance of arguments and detecting evidence, but less so for assessing the quality of arguments. |
Naver Labs Europe’s Systems for the Document-Level Generation and Translation Task at WNGT 2019 (D19-56)
Copied to clipboard
| Challenge: | Recent advances in machine translation and natural language generation have created many challenges in this field especially when context is considered. |
| Approach: | They propose to leverage data from machine translation and natural language generation tasks to do transfer learning between MT, NLG and MT with source-side metadata. |
| Outcome: | The proposed approach outperforms the previous state-of-the-art on the Rotowire NLG task. |
Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce (P18-2)
Copied to clipboard
| Challenge: | Recent researches focus on deep learning and reinforcement learning for multi-turn information seeking conversation systems. |
| Approach: | They propose an efficient and effective multi-turn conversation model based on convolutional neural networks and extend it to adapt the knowledge learned from a resource-rich domain to enhance the performance. |
| Outcome: | The proposed model performs better than the existing model on an industrial chatbot called AliMe Assist. |
Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification (2024.acl-long)
Copied to clipboard
| Challenge: | Existing annotation codebook is labor-intensive for coding events from large datasets. |
| Approach: | They propose to use existing annotation codebook to classify political relations without extensive annotations. |
| Outcome: | The proposed methods outperform dictionary-based methods and the existing ontology annotation codebook and improve interpretability and efficiency. |
Combining Weakly Supervised ML Techniques for Low-Resource NLU (2021.naacl-industry)
Copied to clipboard
| Challenge: | Recent advances in transfer learning have improved the performance of virtual assistants . however, meager training data is often a key bottleneck in creating voice-enabled applications . |
| Approach: | They propose to use unsupervised and semi-supervised techniques to improve NLU accuracy . they incorporate anonymized, unlabeled and automatically transcribed user utterances into training . |
| Outcome: | The proposed methods improve NLU accuracy in low-resource settings by integrating unsupervised and SSL techniques. |
Arcee’s MergeKit: A Toolkit for Merging Large Language Models (2024.emnlp-industry)
Copied to clipboard
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, Jacob Solawetz
| Challenge: | Open-source language models can merge their parameters to improve performance and versatility without additional training. |
| Approach: | They propose to integrate model checkpoints into powerful multitask models without additional training. |
| Outcome: | the library has facilitated the merging of thousands of models, contributing to some of the world’s most powerful open-source model checkpoints. |
FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation struggle to disambiguate rare senses . current models struggle to learn senses with few training examples . |
| Approach: | They introduce a low-shot WSD dataset automatically extracted from example sentences in Wiktionary. |
| Outcome: | The proposed dataset outperforms baseline models on rare senses in existing datasets. |
GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)
Copied to clipboard
Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan
| Challenge: | Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models. |
| Approach: | They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks. |
| Outcome: | The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages. |
Efficient Argument Structure Extraction with Transfer Learning and Active Learning (2022.findings-acl)
Copied to clipboard
| Challenge: | Identifying and understanding the argumentative discourse structure in text has been a critical task in argument mining. |
| Approach: | They propose a context-aware Transformer-based argument structure prediction model that outperforms models that rely on features or only encode limited contexts. |
| Outcome: | The proposed model outperforms models that rely on features or encode limited contexts on five domains and on peer reviews on five different domains. |
ArMATH: a Dataset for Solving Arabic Math Word Problems (2022.lrec-1)
Copied to clipboard
| Challenge: | This paper is the first to use deep learning methods to solve Arabic MWPs . it is also the first study to use transfer learning to solve MWp across different languages . |
| Approach: | They contribute to the first large-scale dataset for Arabic Math Word Problems . they use deep learning methods to solve Arabic MWPs and a transfer learning model to promote performance . |
| Outcome: | The proposed model improves Arabic MWP solvers by 3% over the existing model. |
Weakly-supervised Deep Cognate Detection Framework for Low-Resourced Languages Using Morphological Knowledge of Closely-Related Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to cognate detection focus on orthographic, phonetic or contextual models, which under-perform for most under-resourced languages. |
| Approach: | They propose a language-agnostic weakly-supervised deep cognate detection framework for under-resourced languages using morphological knowledge from closely related languages. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on cognate detection datasets across languages and can be extended to a wide range of languages from any language family. |
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Language models are a new standard to build state-of-the-art NLP systems. |
| Approach: | They compare multilingual and monolingual models on unseen languages . they show that some languages benefit from transfer learning whereas others don't . |
| Outcome: | The proposed model behaves in multiple ways on unseen languages, while others fail to transfer . the results provide a promising direction towards making multilingual models useful for a new set of unseense languages. |
Distilling the Knowledge of Romanian BERTs Using Multiple Teachers (2022.lrec-1)
Copied to clipboard
Andrei-Marius Avram, Darius Catrina, Dumitru-Clementin Cercel, Mihai Dascalu, Traian Rebedea, Vasile Pais, Dan Tufis
| Challenge: | Existing approaches to train pre-trained language models focus on the English language, thus widening the gap when considering low-resource languages. |
| Approach: | They propose three versions of distilled BERT models for the Romanian language . they argue that the models offer performance comparable to their teachers . |
| Outcome: | The proposed models perform comparable to their teachers, while being twice as fast on a GPU and 35% smaller. |
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning (2021.naacl-main)
Copied to clipboard
Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah
| Challenge: | Recent work shows that multilingual representations are disjointed across languages, bringing additional challenges for transfer onto extremely low-resource languages. |
| Approach: | They propose a meta-learning based framework that learns to transform representations judiciously from auxiliary languages to a target one and brings their representation spaces closer for effective transfer. |
| Outcome: | The proposed framework learns to transform representations from auxiliary languages to a target language and brings their representation spaces closer for effective transfer. |
ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cherokee is a highly endangered Native American language spoken by the Cherokee people . there are only 2,000 fluent first language Cherokee speakers remaining in the world . |
| Approach: | They propose a Cherokee-English parallel dataset to facilitate machine translation between Cherokee and English. |
| Outcome: | The proposed dataset compares Cherokee-English and English-Cherokee machine translation systems . the results show that the datasets are low-resource and low-cost compared to other datasets . |
Authorship Attribution of Late 19th Century Novels using GAN-BERT (2023.acl-srw)
Copied to clipboard
| Challenge: | Conventional techniques and neural networks are the two main authorship attribution methods. |
| Approach: | They used a dataset of late 19th century novels in English to fine-tune a transformer-based authorship attribution model using transfer learning. |
| Outcome: | The proposed model outperforms the existing model with 0.88 accuracy and F1 scores. |
Is Prompt Transfer Always Effective? An Empirical Study of Prompt Transfer for Question Answering (2024.naacl-short)
Copied to clipboard
| Challenge: | Prompt tuning is an efficient method for initializing pre-trained models . but initialization of prompts is sensitive when the model size is small . |
| Approach: | They propose a method to measure catastrophic forgetting by analyzing prompts for the first time . they characterize a question answering task based on answer format and prompt initialization . |
| Outcome: | The proposed approach can help deepen understanding of prompt tuning. |
Convolutional Neural Networks for Financial Text Regression (P19-2)
Copied to clipboard
| Challenge: | Recent studies have defined forecasting financial volatility from annual reports as text regression problem. |
| Approach: | They propose to replace word features with word embedding vectors to remove lexicon dependency. |
| Outcome: | The proposed model provides more accurate volatility predictions than lexicon based models. |
Language Models for Code-switch Detection of te reo Māori and English in a Low-resource Setting (2022.findings-naacl)
Copied to clipboard
Jesin James, Vithya Yogarajan, Isabella Shields, Catherine Watson, Peter Keegan, Keoni Mahelona, Peter-Lucas Jones
| Challenge: | Te reo Mori is New Zealand’s only indigenous language spoken by 4.5% of the population of 5 million. |
| Approach: | They train bilingual sub-word embeddings to detect Mori-English code-switching points using a cloud-based multilingual system such as Google and Microsoft Azure. |
| Outcome: | The proposed model outperforms large-scale contextual models on down streaming tasks of detecting Mori language. |
Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for building task-oriented dialog systems are limited to a few tasks and domains. |
| Approach: | They propose a method that uses transfer learning and machine teaching to build task bots at scale. |
| Outcome: | The proposed method outperforms existing methods on well-studied task-oriented dialog benchmarks on well studied tasks. |
On the Use of External Data for Spoken Named Entity Recognition (2022.naacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) tasks require large labeled datasets to perform . compared to prior work, relative improvements in F1 of up to 16% are found . |
| Approach: | They propose to use self-training, knowledge distillation, and transfer learning to learn SLU models . they compare pipeline and pipeline approaches to find out how to use external data . |
| Outcome: | The proposed models improve performance beyond pre-trained models in resource-constrained settings . the best baseline model is a pipeline approach, while the best performance is achieved by an E2E model. |
Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task (N18-1)
Copied to clipboard
| Challenge: | Previously, neural methods in grammatical error correction did not reach state-of-the-art results compared to phrase-based statistical machine translation (SMT) systems that improve on results by SMT use their set-up as a backbone for more complex systems. |
| Approach: | They propose a set of model-independent methods for neural GEC that can be easily applied in most GEC settings. |
| Outcome: | The proposed methods outperform state-of-the-art neural GEC systems by 10% M2 on the CoNLL-2014 benchmark and 5.9% on the JFLEG test set. |
Negation Detection in Dutch Spoken Human-Computer Conversations (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing negation detection methods in English are not available. |
| Approach: | They propose to annotate a Dutch dialogue corpus with negation cues and their scopes. |
| Outcome: | The proposed method can detect negation cues and scope in Dutch dialogues with high precision and recall. |
A Hybrid Approach to Aspect Based Sentiment Analysis Using Transfer Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Aspect-Based Sentiment Analysis (ABSA) aims to identify terms or multiword expressions (MWEs) on which sentiments are expressed and the sentiment polarities associated with them. |
| Approach: | They propose a hybrid approach to Aspect-Based Sentiment Analysis using transfer learning . they exploit the strengths of large language models and traditional syntactic dependencies . |
| Outcome: | The proposed method exploits the strengths of large language models and traditional syntactic dependencies. |
Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks (2022.tacl-1)
Copied to clipboard
| Challenge: | Natural language understanding (NLU) has made massive progress driven by large benchmarks, but a long tail of infrequent phenomena is underrepresented. |
| Approach: | They conceptualize the long tail using macro-level dimensions and perform a meta-analysis of 100 representative papers on transfer learning for NLU. |
| Outcome: | The results highlight avenues for future research in transfer learning for the long tail . authors suggest that the results may be useful for future studies . |
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER (2022.aacl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models can be trained for emerging topics such as medical domain where new topics are constantly evolving out of the scope of existing models and datasets. |
| Approach: | They propose a recipe to combine weak and strong labels to improve Named Entity Recognition (NER) models for emerging topics. |
| Outcome: | The proposed model outperforms methods trained on weak data while combining out-of-domain and in-domain weak label training. |
Syntactic and Semantic Uniformity for Semantic Parsing and Task-Oriented Dialogue Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to model natural language use pre-trained language models, but little attention has been paid to the representation of machine-readable formats. |
| Approach: | They propose a data representation framework for semantic parsing and task-oriented dialogue systems . they define a meta grammar for syntactically uniform representations and translate semantically equivalent functions into a uniform vocabulary. |
| Outcome: | The proposed representation improves accuracy and allows for transfer learning across datasets. |
Multi-task dialog act and sentiment recognition on Mastodon (C18-1)
Copied to clipboard
| Challenge: | Social media are a gold mine for researchers in many domains and especially in natural language processing . license restrictions make it difficult to strictly reproduce research results on Twitter data . |
| Approach: | They propose to annotate a Twitter-like corpus from a decentralized social network with permissive licenses that are compatible with reproducible experiments. |
| Outcome: | The proposed method shows that transfer learning can be efficiently achieved between tasks. |
CogTaskonomy: Cognitively Inspired Task Taxonomy Is Beneficial to Transfer Learning in NLP (2022.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for transfer learning across tasks in natural language processing are CRA and CNM . CRA uses a correlation between brain-activity measurement and computational modeling to estimate task similarity with sentence representations. |
| Approach: | They propose a cognitively inspired framework to learn taxonomy for NLP tasks . they use Cognitive Representation Analytics and Cognitive-Neural Mapping . |
| Outcome: | The proposed framework can guide transfer learning across tasks in natural language processing without exhaustive pairwise task transferring. |
Multilingual Speech Translation from Efficient Finetuning of Pretrained Models (2021.acl-long)
Copied to clipboard
Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli
| Challenge: | Recent advances in text pretraining and finetuning have improved multitasking applications significantly. |
| Approach: | They propose a minimalistic LNA finetuning approach to build multilingual speech-to-text translation using a pretrained speech encoder and text decoder. |
| Outcome: | The proposed approach surpasses the cascaded ST benchmark for 36 translation directions on the large-scale multilingual ST benchmark CoVoST 2. |
Improving Commonsense Contingent Reasoning by Pseudo-data and Its Application to the Related Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Contingent reasoning is one of the essential abilities in natural language understanding . despite advances in deep learning, the task of contingent reasoning is still difficult for computers . |
| Approach: | They propose to generate large-scale pseudo-problems and incorporate them into training . they also investigate the generality of contingent knowledge through quantitative evaluation . |
| Outcome: | The proposed method is able to evaluate the generality of contingent knowledge through transfer learning. |
Exploring Enhanced Code-Switched Noising for Pretraining in Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual pretraining approaches to denoise synthetic code-switched data have shown that they generate the noise using non-contextual, one-to-one word translations obtained from lexicons. |
| Approach: | They propose an approach where contextual, many-to-many word translations are generated using a ‘base’ NMT model. |
| Outcome: | The proposed approach improves on 3 different language families and shows that small models can perform better than massive models like mBART50 and mRASP2 . |
Text-to-Code Generation with Modality-relative Pre-training (2024.eacl-long)
Copied to clipboard
| Challenge: | Large pre-trained language models have been applied to programming language tasks with great success, often through further pre-training of a strictly-natural language model. |
| Approach: | They propose to map programming language modalities into the same embedding space by separating embeddable spaces between modality and modality-relative training objectives. |
| Outcome: | The proposed model can be adapted and represented differently depending on which modality they belong to and to the ultimate benefit of the downstream task. |
Progressive Sentiment Analysis for Code-Switched Text Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual transformer language models are used in cross-lingual transfer learning for many NLP tasks such as text classification and named entity recognition. |
| Approach: | They propose a framework that takes the distinction between resource-rich and low-resource language into account and progressively trains from resource-dominated to low-rsource samples. |
| Outcome: | The proposed model outperforms existing methods on low-resource languages and performs well on medium-resourced to high-res languages. |
NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classification (2023.acl-short)
Copied to clipboard
| Challenge: | Africa has over 2000 indigenous languages but they are under-represented in NLP research due to lack of datasets. |
| Approach: | They propose to use a dataset to classify sentiments for cross-domain adaptation for Nigerian and other African languages. |
| Outcome: | The proposed dataset compares the performance of cross-domain adaptation from Twitter domain and cross-lingual adaptation from English domain. |
Context-Aware Text Normalisation for Historical Dialects (2020.coling-main)
Copied to clipboard
| Challenge: | Context-aware historical text normalisation is a severely under-researched area . a new approach to normalise historical spellings relies on the state-of-the-art methods . |
| Approach: | They propose a multidialect normaliser with a context-aware reranking approach . they incorporate dialectal information into the training and use a word-level n-gram language model . |
| Outcome: | The proposed approach improves accuracy on historical datasets and further improves on baseline. |
AMUSED: A Multi-Stream Vector Representation Method for Use in Natural Dialogue (2020.lrec-1)
Copied to clipboard
| Challenge: | Current architectures only take care of semantic and contextual information for a given query and fail to fully account for syntactic and external knowledge which are crucial for generating responses in a chit-chat system. |
| Approach: | They propose a multi-stream deep learning architecture that learns unified embeddings for query-response pairs by incorporating Graph Convolution Networks over their dependency parse. |
| Outcome: | The proposed architecture improves on the next sentence prediction task and significantly improves existing techniques. |
AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation (2021.acl-long)
Copied to clipboard
| Challenge: | Large-scale conversational systems typically generate unnatural, robotic responses using template-based approaches. |
| Approach: | They propose a data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model to automatically create MR-to-Text data from open-domain texts. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods on the FewshotWOZ data in both BLEU and Slot Error Rate. |
Predicting Foreign Language Usage from English-Only Social Media Posts (N18-2)
Copied to clipboard
| Challenge: | Social media is known for its multi-cultural and multilingual interactions, a natural product of which is code-mixing. |
| Approach: | They analyze 6 million tweets produced by 27 thousand multilingual users speaking 12 other languages besides English to build predictive models to infer non-English languages users speak exclusively from their tweets. |
| Outcome: | The proposed models are based on a corpus of 6 million tweets produced by 27 thousand multilingual users speaking 12 other languages besides English . they show that content, style and syntax are the most predictive of non-English languages that users speak on Twitter. |
Robust Transfer Learning with Pretrained Language Models through Adapters (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to transfer learning with pretrained transformer-based language models are not robust and can be adversarial. |
| Approach: | They propose a simple yet effective adapter-based approach to fine-tune language models on downstream tasks. |
| Outcome: | The proposed approach improves stability and adversarial robustness in transfer learning to various downstream tasks. |
Exploring Neural Models for Query-Focused Summarization (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work in Query-focused summarization lacks a comprehensive study of the broad space of applicable modeling methods. |
| Approach: | They propose to explore two general classes of methods for Query-focused summarization: extractive-abstractive solutions and end-to-end models. |
| Outcome: | The proposed models achieve state-of-the-art on the QMSum dataset, with a margin of 3.38 ROUGE-1, 3.72 ROUGe2 and 3.28 ROUGEL-L. |
CVAE-based Re-anchoring for Implicit Discourse Relation Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training implicit discourse relation classifiers suffers from data sparsity. |
| Approach: | They propose a re-anchoring strategy to reduce the risk of erroneous sampling . they use Conditional VAE to estimate the risk and migrate the anchor to reduce it . |
| Outcome: | The proposed method improves the baseline classifier performance on PDTB v2.0 . |
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing (2024.naacl-long)
Copied to clipboard
| Challenge: | Standard fine-tuning of language models suffers with generalization to distribution shifts. |
| Approach: | They propose to use Fisher Information to investigate scheduled unfreezing algorithms for adapter-based cross-lingual task transfer to improve generalization to distribution shifts. |
| Outcome: | The proposed method achieves an average of 2 points improvement over four datasets compared to standard fine-tuning and provides empirical evidence for a theory-based justification of the proposed method. |
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
Pruning Adatperfusion with Lottery Ticket Hypothesis (2022.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained language models are computationally expensive to fine-tune and require large storage. |
| Approach: | They propose a method to identify the influence of each adapter module and a way to prune adapters based on the Lottery Ticket Hypothesis. |
| Outcome: | The proposed model reduces size significantly while keeping performance intact. |
Dynamically Composing Domain-Data Selection with Clean-Data Selection by “Co-Curricular Learning” for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing studies focus separately on domain-data selection, clean-data selecting, or their static combination, leaving the dynamic interaction across them not explicitly examined. |
| Approach: | They propose a method to combine dynamic domain-data selection with dynamic clean-data selecting for transfer learning across both capabilities. |
| Outcome: | The proposed method performs well on two domains and shows the properties of the data scheduled by the co-curriculum. |
Towards Difficulty-Agnostic Efficient Transfer Learning for Vision-Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Vision-language models (VLMs) have demonstrated remarkable applicability across downstream tasks, including zero-shot image classification. |
| Approach: | They propose an efficient transfer learning method that integrates visual prompts and text adapters with pre-trained VLMs to achieve optimal performance for any target domain. |
| Outcome: | The proposed method outperforms baselines on unseen tasks. |
Investigating Transferability in Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on deep NLP models has centered on probing, a method that involves training classifiers for different tasks on model representations. |
| Approach: | They propose a method for determining the impact of each pretrained layer on transfer task performance by ablation. |
| Outcome: | The proposed method shows that pretraining models improve performance on downstream tasks . the results highlight the limitations of methods that operate on frozen models or single data samples. |
Composable Sparse Fine-Tuning for Cross-Lingual Transfer (2022.acl-long)
Copied to clipboard
| Challenge: | Adapters and sparse fine-tuning have been developed to improve transfer learning . a number of approaches have been proposed to improve performance of fine-untuners . |
| Approach: | They propose a method that fine-tunes the entire set of parameters of a large pretrained model . they use adapters and sparse fine-uning to improve model efficiency . |
| Outcome: | The proposed method outperforms adapters in cross-lingual transfer benchmarks. |
Transfer Learning for a Letter-Ngrams to Word Decoder in the Context of Historical Handwriting Recognition with Scarce Resources (C18-1)
Copied to clipboard
| Challenge: | Lack of data can be an issue when beginning a new study on historical handwritten documents. |
| Approach: | They propose a character-based decoder for historical handwriting recognition on Italian Comedy Registers . they use untapped data from domains, periods, languages to obtain efficient system . |
| Outcome: | The character-based decoder can be used to learn historical handwriting on Italian Comedy registers . the results show that the system can be obtained by carefully selecting the datasets used . |
Can We Guide a Multi-Hop Reasoning Language Model to Incrementally Learn at Each Single-Hop? (2022.coling-1)
Copied to clipboard
| Challenge: | Recent developments have shown that pre-trained language models are effective soft reasoners over language. |
| Approach: | They propose to model multi-hop reasoning process as a sequence of explicit single-hop steps. |
| Outcome: | The proposed model improves on multiple-choice question answering and reading comprehension with 68.4% and 16.0% w.r.t. classic PLMs. |
Knowledge Augmentation Enhances Token Classification for Recipe Understanding (2026.eacl-long)
Copied to clipboard
| Challenge: | Using entity type-specific and knowledge-augmented token classification, we achieve state-of-the-art (SOTA) results on 5 out of 7 benchmark recipe datasets, significantly outperforming traditional token classification methods. |
| Approach: | They propose an entity type-specific and knowledge-augmented token classification framework to improve encoder models’ performance on recipe texts. |
| Outcome: | The proposed model outperforms traditional token classification methods on 5 out of 7 recipe datasets and is the largest annotated food-related dataset to date. |
RuCCoD: Towards Automated ICD Coding in Russian (2025.emnlp-main)
Copied to clipboard
Alexandr Nesterov, Andrey Sakhovskiy, Ivan Sviridov, Airat Valiev, Vladimir Makharev, Petr Anokhin, Galina Zubkova, Elena Tutubalina
| Challenge: | a new dataset for clinical coding in Russian is available for download . human coders must navigate a wide array of medical terminology and time pressures . |
| Approach: | They present a new dataset for ICD coding in Russian, a language with limited biomedical resources. |
| Outcome: | The proposed model improves accuracy on an in-house EHR dataset from 2017 to 2021. |
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a series of experiments show that fine-tuning only the cross-attention parameters is nearly as effective as fine-timing all parameters. |
| Approach: | They conduct experiments to fine-tune a translation model on data where either the source or target language has changed. |
| Outcome: | The proposed model can be trained to several new languages with reduced parameter storage overhead. |
Bridging Anaphora Resolution as Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on bridging anaphora resolution focus on question answering based on context . briding anaphorisms and their antecedents are linked via various lexico-semantic, frame or encyclopedic relations. |
| Approach: | They propose a question answering framework for bridging anaphora resolution . they propose briding anaphorisms and their antecedents are linked via various lexico-semantic, frame or encyclopedic relations. |
| Outcome: | The proposed method generates state-of-the-art results on two bridging corpora. |
Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices (2025.emnlp-industry)
Copied to clipboard
Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Mizanur Rahman, Amran Bhuiyan, Israt Jahan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang
| Challenge: | Large Vision-Language Models (LVLMs) with only 7B parameters perform poorly as judges in resource-constrained settings. |
| Approach: | They propose two approaches to ensure costefficient evaluation by combining multiple criteria into a single query and domainadaptive transfer learning to create a 2Bparameter VLM on a chart dataset. |
| Outcome: | The proposed model can effectively transfer knowledge from one dataset to another to make it a more specialized model. |
Cross Encoding as Augmentation: Towards Effective Educational Text Classification (2023.findings-acl)
Copied to clipboard
Hyun Seung Lee, Seungtaek Choi, Yunsung Lee, Hyeongdon Moon, Shinhyeok Oh, Myeongho Jeong, Hyojun Go, Christian Wallraven
| Challenge: | Existing methods to improve text classification in education suffer from data scarcity . authors propose a retrieval approach that provides effective learning in educational text classification. |
| Approach: | They propose a retrieval approach that provides effective learning in educational text classification by introducing cross-encoder style texts to a bi-encoding architecture. |
| Outcome: | The proposed method is effective in multi-label scenarios and low-resource tags compared to state-of-the-art models. |
Asking and Answering Questions to Extract Event-Argument Structures (2024.lrec-main)
Copied to clipboard
| Challenge: | Traditionally, corpora are limited to arguments within the same sentence, and inter-sentential arguments are more challenging and have received less attention. |
| Approach: | They propose a question-answering approach to extract document-level event-argument structures by automating questions for each argument type an event may have. |
| Outcome: | The proposed model outperforms previous models and is especially beneficial to extract arguments that appear in different sentences than the event trigger. |
Supervised and Unsupervised Transfer Learning for Question Answering (N18-1)
Copied to clipboard
| Challenge: | Several QA scenarios and datasets have been introduced over the past few years. |
| Approach: | They conduct extensive experiments to investigate the transferability of knowledge from a source QA dataset to a target dataset using two QA models. |
| Outcome: | The proposed model outperforms the previous best model on TOEFL listening comprehension test by 7% on target datasets. |
Cross-lingual Contextualized Topic Models with Zero-shot Learning (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing topic models are language-specific and cannot be transferred in a transferable manner. |
| Approach: | They propose a zero-shot cross-lingual topic model that learns topics on one language and predicts them for unseen documents in different languages. |
| Outcome: | The proposed model learns topics on one language and predicts them for unseen documents in different languages. |
Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates (2021.eacl-main)
Copied to clipboard
Artem Shelmanov, Dmitri Puzyrev, Lyubov Kupriyanova, Denis Belyakov, Daniil Larionov, Nikita Khromov, Olga Kozlova, Ekaterina Artemova, Dmitry V. Dylov, Alexander Panchenko
| Challenge: | Annotating training data for sequence tagging of texts is usually very time-consuming . active learning can help to reduce the amount of annotation required to train a good model by multiple times . |
| Approach: | They are the first to thoroughly investigate active learning and transfer learning for natural language processing . they propose to combine active learning with active learning to improve model acquisition . |
| Outcome: | The proposed combination of active learning and Bayesian uncertainty estimation improves performance and reduces obstacles for applying it in practice. |
IndicBART: A Pre-trained Model for Indic Natural Language Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | IndicBART is a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. |
| Approach: | They present a multilingual sequence-to-sequence pre-trained model for Indic languages . they evaluate it on two NLG tasks: Neural Machine Translation and extreme summarization . |
| Outcome: | The proposed model performs well on low-resource translation scenarios . Script sharing, multilingual training, and better utilization contribute to the performance. |
Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Using multi-parallel corpora for transfer learning is a useful technique for low-resource NMT. |
| Approach: | They compare multi-parallel corpora for transfer learning in a low-resource setting . their results show that multi-paralleled corpors are extremely useful . |
| Outcome: | The proposed model can give 3–9 BLEU score gains over a one-to-one model. |
Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering (D19-1)
Copied to clipboard
| Challenge: | Object detection is used in vision and language tasks but is expensive to learn . popular models rely on annotating ground-truths for bounding boxes and semantic labels . empirically, object detection leads to effective transfer learning and improved captioning and visual question answering models . |
| Approach: | They examine the effect of decoupling box proposal and featurization on down-stream tasks . they propose a family of "two-stage" object detectors that propose category-agnostic bounding boxes . |
| Outcome: | The proposed method improves image captioning and visual question answering models by leveraging large amounts of labeled annotations. |
Towards Few-shot Fact-Checking via Perplexity (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models can perform few-shot learning for various downstream tasks, such as question answering and machine translation. |
| Approach: | They propose a method to leverage the powerful transfer learning ability of a language model via a perplexity score to learn few-shot for the fact-checking task. |
| Outcome: | The proposed method outperforms the Major Class baseline by 10% on the F1-Macro metric across multiple datasets. |
Active Learning Approaches to Enhancing Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a limited human translation budget is required to train neural machine translation models. |
| Approach: | They propose to integrate active learning into neural machine translation techniques . they propose a word frequency based acquisition function and an uncertainty based method . |
| Outcome: | The proposed method outperforms other acquisition functions on a limited human translation budget. |
Japanese Zero Anaphora Resolution Can Benefit from Parallel Texts Through Neural Transfer Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Using a pretraining model, we find that the performance of Japanese zero anaphora resolution (ZAR) is improved by using machine translation. |
| Approach: | They propose to inject machine translation as an intermediate task between pretraining and ZAR by injecting machine translation into a pretrained BERT model and injecting it into MT. |
| Outcome: | The proposed framework shows that Japanese zero anaphora resolution (ZAR) can be improved by transfer learning from machine translation (MT). |
Does Masked Language Model Pre-training with Artificial Data Improve Low-resource Neural Machine Translation? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pre-training masked language models with artificial data has been proven beneficial for several natural language processing tasks, however, it has been less explored for neural machine translation (NMT). |
| Approach: | They pre-trained masked language models with random sequences and created artificial data mimicking token frequency information from the real world. |
| Outcome: | The results show that pre-training models with artificial data improves translation performance in low-resource situations. |
Investigating Multilingual NMT Representations at Scale (D19-1)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models have shown success in transfer learning settings, but their mode of transfer remains elusive. |
| Approach: | They propose to use a representation similarity framework to compare multilingual representations using a SVCCA representation similar to the previous work. |
| Outcome: | The proposed model can be used to compare representations across languages and layers. |
Transfer Learning for Entity Recognition of Novel Classes (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to entity recognition are based on class labels in source and target domains, and many NER corpora only annotate a small number of categories. |
| Approach: | They replicate and extend several past studies on transfer learning for entity recognition. |
| Outcome: | The proposed methods perform better when there is more labeled target data. |
Transfer Learning from Semantic Role Labeling to Event Argument Extraction with Template-based Slot Querying (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event argument extraction are limited by the limited amount of annotations available. |
| Approach: | They propose to use SRL annotations for event argument extraction . they propose to specify natural language-like queries to tackle label mismatch problem . |
| Outcome: | The proposed model achieves impressive zero-shot results on English benchmarks . it also provides benefits in low-resource cases, where few annotations are available . |
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies (2021.eacl-main)
Copied to clipboard
| Challenge: | Biomedical question-answering (QA) provides users with high-quality information from a vast scientific literature. |
| Approach: | They propose to use a biomedical entity-aware masking strategy to fine-tune masked language models to their domains. |
| Outcome: | The proposed approach is an adaptation process for masked LMs, not memory or components. |
Hierarchical Recurrent Aggregative Generation for Few-Shot NLG (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches do not account for the fact that some sub-tasks, specifically aggregation and lexicalisation, can benefit from transfer learning in different extents. |
| Approach: | They propose a hierarchical approach for few-shot and zero-shot generation using a three-moduled jointly trained architecture. |
| Outcome: | The proposed approach achieves state-of-the-art on few-shot and zero-shot settings compared to previous approaches. |
An Evaluation of Progressive Neural Networksfor Transfer Learning in Natural Language Processing (2020.lrec-1)
Copied to clipboard
Abdul Moeed, Gerhard Hagerer, Sumit Dugar, Sarthak Gupta, Mainak Ghosh, Hannah Danner, Oliver Mitevski, Andreas Nawroth, Georg Groh
| Challenge: | Fine-tuning suffers from catastrophic forgetting, a problem exacerbated in natural language processing (NLP). |
| Approach: | They propose to use progressive neural networks to re-use previously learned knowledge when learning new tasks. |
| Outcome: | The proposed approach improves on common NLP tasks across a range of architectures, datasets, and tasks. |
Offensive Language Detection on Video Live Streaming Chat (2020.coling-main)
Copied to clipboard
| Challenge: | a prototype of a live chat room that detects offensive expressions in live streaming chats is presented . offensive expression detection on social media platforms can provide more protection for users . |
| Approach: | They propose a live chat room that detects offensive expressions in live streaming chats in real time . they used a dataset from Twitch to analyze offensive expression patterns . |
| Outcome: | The proposed chat room detects offensive expressions in live streaming chats in real time. |
Transfer and Multi-Task Learning for Noun–Noun Compound Interpretation (D18-1)
Copied to clipboard
| Challenge: | In computational linguistics, nounnoun compound interpretation is approached as an automatic classification problem. |
| Approach: | They empirically evaluate the utility of transfer and multi-task learning on a challenging semantic classification task. |
| Outcome: | The proposed methods improve the accuracy of a neural classifier and its F1 scores on the less frequent, but more difficult relations. |
A Workflow for HTR-Postprocessing, Labeling and Classifying Diachronic and Regional Variation in Pre-Modern Slavic Texts (2024.lrec-main)
Copied to clipboard
Piroska Lendvai, Maarten van Gompel, Anna Jouravel, Elena Renje, Uwe Reichel, Achim Rabus, Eckhart Arnold
| Challenge: | a workflow for classifying diachronic and regional language variation in medieval texts is currently being developed . the workflow is generic or language-agnostic, but can be applied to other historical languages as well. |
| Approach: | They propose a workflow for classifying diachronic and regional language variation in medieval texts . they use handwritten text recognition and manual transcription to obtain the data . |
| Outcome: | The proposed workflow covers HTR-postprocessing, annotating and classifying medieval texts . it is accessible to humanists with limited experience in research data infrastructures, analysis or NLP . |
Lessons from Natural Language Inference in the Clinical Domain (D18-1)
Copied to clipboard
| Challenge: | State of the art models with deep neural networks lack generalization capabilities in specialized domains where training data is limited. |
| Approach: | They propose a dataset annotated by doctors performing a natural language inference task grounded in the medical history of patients. |
| Outcome: | The proposed model outperforms existing models in the clinical domain by incorporating domain knowledge from external data and lexical sources. |
UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot dialogue state tracking (DST) ignore unlabelled data in the target domain. |
| Approach: | They propose to transform zero-shot dialogue state tracking into few-shot DST by utilising unlabelled data via joint and self-training methods. |
| Outcome: | The proposed method improves joint goal accuracy by 8% on general language models in zero-shot scenarios, and can be used in many domains. |
Hate-Speech and Offensive Language Detection in Roman Urdu (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on hate-speech and offensive language detection in social media content is mainly focused on the English language. |
| Approach: | They propose to use an annotated dataset to detect hate-speech and offensive language in social media content . they propose to transfer five existing embedding models to Roman Urdu to test their performance . |
| Outcome: | The proposed model outperforms existing methods on RUHSOLD dataset and train domain-specific embeddings on more than 4.7 million tweets. |
Zero-Shot Rationalization by Multi-Task Transfer Learning from Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract rationales from input text are difficult and impractical. |
| Approach: | They propose a method that leverages multi-task learning and transfer learning to generate rationales through question answering in a zero-shot fashion. |
| Outcome: | The proposed method achieves comparable or even better performance without supervised signal for two benchmark rationalization datasets. |
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | a growing body of work is focused on improving performance in low-resource settings . a goal of this study is to explain how these methods differ in their requirements . |
| Approach: | They propose to analyze data-lean scenarios across different dimensions of data availability to understand which approaches are effective in a specific low-resource setting. |
| Outcome: | The proposed methods enable learning when training data is sparse. |
A Novel Two-step Fine-tuning Framework for Transfer Learning in Low-Resource Neural Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing transfer learning methods for neural machine translation use a well-trained translation model to initialize a child model with corresponding datasets. |
| Approach: | They propose a two-step fine-tuning framework for transfer learning in low-resource neural machine translation that adjusts the parent model to fit the child language by using the child source data. |
| Outcome: | The proposed framework improves on five low-resource translations on high-resolution languages. |
Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that results from high-resource languages cannot be easily transferred to realistic, low-resourced scenarios. |
| Approach: | They analyse performance of multilingual transformer models using available resources for Hausa, isiXhosa and NER and topic classification. |
| Outcome: | The proposed models can achieve with as little as 10 or 100 labeled sentences the same performance as baselines with much more supervised training data. |
Cross-domain Named Entity Recognition via Graph Matching (2022.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show that our method outperforms a series of transfer learning, multitask learning, and few-shot learning methods due to the data scarcity in the real-world scenario. |
| Approach: | They propose to model the label relationship as a probability distribution and construct label graphs in both source and target label spaces. |
| Outcome: | Empirical results show that the proposed method outperforms transfer learning, multi-task learning, and few-shot learning methods on four datasets. |
Gromov-Wasserstein Alignment of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Current unsupervised methods for learning cross-lingual correspondences involve multiple steps, including heuristic post-hoc refinement strategies. |
| Approach: | They propose to cast the correspondence problem directly as an optimal transport problem, building on the idea that word embeddings arise from metric recovery algorithms. |
| Outcome: | The proposed method can be estimated efficiently, requires little or no tuning, and performs comparable with the state-of-the-art in various unsupervised word translation tasks. |
Cross-lingual Continual Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing multi-lingual representations such as the one-hop transfer learning pipeline are difficult to adapt to new languages. |
| Approach: | They propose a cross-lingual continuum learning paradigm that evaluates continuous learning approaches that adapt to emerging data from different languages. |
| Outcome: | The proposed model can be used to adapt to new languages in a sequential manner. |
Improvement in Sign Language Translation Using Text CTC Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Current sign language translation (SLT) approaches rely on gloss-based supervision with Connectionist Temporal Classification (CTC) limiting their ability to handle non-monotonic alignments between sign language video and spoken text. |
| Approach: | They propose a method that integrates CTC/Attention with the attention mechanism during decoding and integrates it with the sign language video and spoken text. |
| Outcome: | The proposed method outperforms the pure-attention baseline and achieves comparable results to state-of-the-art methods. |
Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Currently, most work on targeted sentiment analysis is focused on improving the overall results. |
| Approach: | They propose a multi-task learning method to incorporate information from syntactic and semantic auxiliary tasks to create English-language models that are more robust to linguistic phenomena. |
| Outcome: | The proposed method improves on negation and speculation datasets but there is room for improvement. |
Unregulated Chinese-to-English Data Expansion Does NOT Work for Neural Event Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that cross-language data expansion results in performance degradation. |
| Approach: | They leverage cross-language data expansion and retraining to enhance neural Event Detection on English ACE corpus. |
| Outcome: | The proposed method improves ED performance by 1.6% over the straight data combination. |
Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity? (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing literature has focused on pretrainer-based text-driven brain encoding models . however, few studies have explored the efficacy of task-specific learning of Transformers . |
| Approach: | They propose to use ten popular natural language processing tasks to learn Transformer representations for predicting brain responses. |
| Outcome: | The proposed model predicts brain activity across the whole brain. |
Large-Scale Hate Speech Detection with Cross-Domain Transfer (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for hate speech detection are limited due to the labor cost. |
| Approach: | They construct large-scale tweet datasets for hate speech detection in English and a low-resource language, Turkish, consisting of human-labeled 100k tweets per each. |
| Outcome: | The proposed datasets outperform conventional bag-of-words and neural models by at least 5% in English and 10% in Turkish for large-scale hate speech detection. |
Low Resource Sequence Tagging using Sentence Reconstruction (2020.acl-main)
Copied to clipboard
| Challenge: | This work revisits the task of training sequence tagging models with limited resources using transfer learning. |
| Approach: | They propose a new loss that relies on sentence reconstruction from normalized embeddings. |
| Outcome: | The proposed method achieves 0.6 F1 score on Dutch datasets using only one sample. |
Two Parents, One Child: Dual Transfer for Low-Resource Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation suffers when parallel data is scarce for training . a new framework to transfer multiple sources of auxiliary data is proposed . |
| Approach: | They propose a framework to transfer multiple sources of auxiliary data from high-resource parallel data to low-resourced translation models using pretrained language models. |
| Outcome: | The proposed approach yields consistent improvements over strong competitors for multiple translation directions. |
Neural Relation Extraction via Inner-Sentence Noise Reduction and Transfer Learning (D18-1)
Copied to clipboard
| Challenge: | Existing methods for extracting relations are slow and lack precision . a novel approach to extract relations is proposed to reduce noise between sentences . |
| Approach: | They propose a word-level distant supervised approach for relation extraction using New York Times and Freebase. |
| Outcome: | The proposed method improves the area of precision/call(PR) from 0.35 to 0.39 over the state-of-the-art methods. |
How emotional are you? Neural Architectures for Emotion Intensity Prediction in Microblogs (C18-1)
Copied to clipboard
| Challenge: | Social media based micro-blogging sites like Twitter are used for expressing emotions and opinions. |
| Approach: | They propose to combine convolutional and fully connected layers in a non-sequential manner to train deep multi-task learning models trained for all emotions at once in unified architecture. |
| Outcome: | The proposed model outperforms the previous system by 0.044 or 4.4% on the WASSA’17 EmoInt shared task dataset. |
Celebrity Profiling (P19-1)
Copied to clipboard
| Challenge: | Using a corpus of 71,706 verified accounts, we construct a profile of a wide cross-section of local and global celebrities. |
| Approach: | They propose to use Twitter feeds of 71,706 verified accounts to build a corpus of celebrity profiles using Wikidata crawling. |
| Outcome: | The proposed corpus contains an average of 29,968 words per profile and up to 239 pieces of personal information. |
On the Use of Bert for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained models have not been used to outperform other deep learning models such as CNN in Automated Essay Scoring (AES). |
| Approach: | They propose a novel multi-scale essay representation for BERT that can be jointly learned . they employ multiple losses and transfer learning from out-of-domain essays to further improve performance . |
| Outcome: | The proposed model outperforms existing models in the area of automated essay scoring . the proposed model generalizes well to the CommonLit Readability Prize data set . |
ATFormer: A Learned Performance Model with Transfer Learning Across Devices for Deep Learning Tensor Programs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Compilation-based methods with performance models have poor measurement accuracy and transferability between platforms. |
| Approach: | They propose a compiler that automatically generates tensors and automatically tunes them for different hardware platforms. |
| Outcome: | The proposed model reduces inference time and costs on modern DNN benchmarks. |
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer (2020.acl-main)
Copied to clipboard
| Challenge: | Multilingual word embeddings embed words from many languages into a single semantic space such that words with similar meanings are close to each other regardless of the language. |
| Approach: | They propose to use multilingual word embeddings to align embeddable words from multiple languages into a single semantic space so that words with similar meanings are close to each other regardless of the language. |
| Outcome: | The proposed model can be used to learn gender bias in multilingual representations and to improve transfer learning. |
Challenges in Pre-Training Graph Neural Networks for Context-Based Fake News Detection: An Evaluation of Current Strategies and Resource Limitations (2024.lrec-main)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) are used to train neural networks to detect fake news based on context-based methods. |
| Approach: | They propose to combine the two by applying pre-training of Graph Neural Networks (GNNs) in the domain of context-based fake news detection. |
| Outcome: | The proposed methods show that transfer learning does not lead to significant improvements over training a model from scratch in the domain of context-based fake news detection. |
Estimating the influence of auxiliary tasks for multi-task learning of sequence tagging tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Multitask learning and transfer learning are techniques to overcome data scarcity . finding suitable auxiliary datasets for multitask learning is a trial-and-error approach . |
| Approach: | They propose to automatically assess the similarity of sequence tagging datasets to identify beneficial auxiliary data for MTL or TL setups. |
| Outcome: | The proposed methods can compute similarity between two sequence tagging datasets . they show that the same measures correlate with the change in test score of the auxiliary dataset . |
Domain Adapted Distant Supervision for Pedagogically Motivated Relation Extraction (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing systems for question generation based on shallow linguistic information at text level generate questions for reading comprehension. |
| Approach: | They propose a relation extraction system that extracts pedagogically motivated relation types from a text to generate questions for reading comprehension. |
| Outcome: | The proposed system extracts pedagogically motivated relations from a text . it performs better on the subset of relations suitable for pedagical purposes. |
What does it take to bake a cake? The RecipeRef corpus and anaphora resolution in procedural text (2022.findings-acl)
Copied to clipboard
| Challenge: | Current research on anaphora resolution is mostly based on declarative text, such as chemical patents or instruction manuals. |
| Approach: | They propose a framework for anaphora annotation for the chemical domain for modeling anamorphic phenomena in recipes and chemical patents. |
| Outcome: | The proposed framework improves resolution of anaphora in recipes, suggesting transferability of general procedural knowledge. |
X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have gained popularity for their zero-shot cross-lingual transfer learning capabilities, but their generalization ability is inconsistent for typologically diverse languages. |
| Approach: | They propose a meta-learning approach that adapts MAML to learn to adapt to new languages . they extensively evaluate two cross-lingual NLU tasks using English as source and spanish as target . |
| Outcome: | The proposed approach outperforms naive fine-tuning on cross-lingual tasks for most languages. |
Pushing the Limits of AMR Parsing with Self-Learning (2020.findings-emnlp)
Copied to clipboard
Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Revanth Gangi Reddy, Radu Florian, Salim Roukos
| Challenge: | Abstract Meaning Representation (AMR) parsing has experienced a notable growth in performance in the last two years due to the impact of transfer learning and the development of novel architectures specific to AMR. |
| Approach: | They propose to use AMR annotations to generate synthetic text and refine actions oracle without additional human annotations for AMR parsing. |
| Outcome: | The proposed models improve on AMR 1.0 and 2.0 without human annotations. |
Mapping probability word problems to executable representations (2021.emnlp-main)
Copied to clipboard
Simon Suster, Pieter Fivez, Pietro Totis, Angelika Kimmig, Jesse Davis, Luc de Raedt, Walter Daelemans
| Challenge: | a recent paper addresses the problem of solving math word problems automatically . a number of approaches have been proposed for solving word problems . |
| Approach: | They employ a sequence-to-sequence model to generate intermediate representations for word problems . they then use a probabilistic programming system to provide the answer . their best performing model incorporates general-domain contextualised word representations . |
| Outcome: | The proposed model is the best performing on a declarative language and a probabilistic programming system. |
Adaptation of Deep Bidirectional Transformers for Afrikaans Language (2020.lrec-1)
Copied to clipboard
| Challenge: | Pretrained language models can be trained in unsupervised manner, but can be difficult to implement because of the amount of data and computational resources needed for pretraining. |
| Approach: | They propose a model for Afrikaans based on bidirectional encoder representation from transformers. |
| Outcome: | The proposed model outperforms the existing models in part-of-speech tagging, named-entity recognition, and dependency parsing tasks. |
Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to train high-quality NMT models in bilingually low-resource scenarios are limited by the scarcity of parallel sentence-pairs. |
| Approach: | They propose to distill the knowledge of teacher models to a single student model by using knowledge distillation. |
| Outcome: | The proposed approach achieves up to +0.9 BLEU score improvements compared to strong baselines. |
Federated Learning for Spoken Language Understanding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to improve robustness of models focus on a single dataset . but, there are few studies on how to combine merits of different datasets . |
| Approach: | They propose a federated learning framework that could unify datasets and tasks . they propose MV-Encoder as backbone of the framework to provide multi-granularity text representations . |
| Outcome: | The proposed framework improves on two SLU benchmark datasets and federated learning settings. |
ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)
Copied to clipboard
Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan
| Challenge: | Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills. |
| Approach: | They propose a unified QA paradigm that solves various tasks through a single model. |
| Outcome: | The proposed model improves QA-centric ability on 11 QA benchmarks. |
Domain Adaptation of Thai Word Segmentation Models using Stacked Ensemble (2020.emnlp-main)
Copied to clipboard
Peerat Limkonchotiwat, Wannaphong Phatthiyaphaibun, Raheem Sarwar, Ekapol Chuangsuwanich, Sarana Nutanong
| Challenge: | Thai word segmentation is domain-dependent, and researchers have been relying on transfer learning to adapt existing models to new domains. |
| Approach: | They propose a filter-and-refine solution to address Thai word segmentation as a domain-dependent problem. |
| Outcome: | The proposed method is an effective domain adaptation method and has similar performance as the transfer learning method. |
FaLA: Fast Linear Adaptation for Replacing Backbone Models on Edge Devices (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current NLP models heavily rely on pre-trained models, such as BERT and RoBERTa. |
| Approach: | They propose a lightweight method for personalized NLP classification tasks post-backbone replacement using a personalized matrix calculated from documents corresponding to users' old and new backbones. |
| Outcome: | The proposed method achieves over 1000 times computation reduction in Flops for backpropagation and brings the user-specific initialization for personal matrix yielding significant performance boost compared with popular transfer learning methods. |
The Woman Worked as a Babysitter: On Biases in Language Generation (D19-1)
Copied to clipboard
| Challenge: | a systematic study of biases in natural language generation (NLG) is presented . a study of language models in NLG is conducted by examining language models. |
| Approach: | They propose a systematic study of biases in natural language generation by analyzing text generated from prompts that contain mentions of different demographic groups. |
| Outcome: | The proposed method reveals biases in natural language generation (NLG) by analyzing text generated from demographic prompts. |
Multi-Aspect Transfer Learning for Detecting Low Resource Mental Disorders on Social Media (2022.lrec-1)
Copied to clipboard
| Challenge: | Mental disorders are an important and pervasive public health issue. |
| Approach: | They propose to use linguistic features to improve mental disorder detection . they propose to apply multi-aspect transfer learning to detecting disorders from social media . |
| Outcome: | The proposed methods can be used to improve mental disorder detection in the context of data scarcity and understanding the overlapping symptoms between disorders. |
Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evidence of demographic bias in SA systems is limited to a handful of languages, and it is costly to create supervised training data in a new language. |
| Approach: | They use counterfactual evaluation to test whether gender or racial biases are imported when using cross-lingual transfer . r&r is much more prevalent than gender biase . |
| Outcome: | The proposed model is compared with monolingual systems in five languages and shows that it is biased more than monolingual ones. |
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion (N19-1)
Copied to clipboard
| Challenge: | Existing datasets for sentence fusion are small and insufficient for training modern neural models. |
| Approach: | They propose a method for automatically-generating fusion examples from raw text . they apply their method to Wikipedia and Sports articles to generate fusion models . |
| Outcome: | The proposed method improves performance on WebSplit when viewed as a sentence fusion task. |
Distilling Knowledge from Text-to-Image Generative Models Improves Visio-Linguistic Reasoning in CLIP (2024.emnlp-main)
Copied to clipboard
| Challenge: | Image-text contrastive models like CLIP struggle on compositional visio-linguistic tasks where their performance is no better than random chance. |
| Approach: | They propose a distillation method to enhance CLIP's compositional visio-linguistic reasoning by using a model-derived distillation objective borrowed from large text-to-image generative models like Stable-Diffusion. |
| Outcome: | The proposed method improves CLIP models' visio-linguistic performance on the Winoground benchmark by 7% while on the ARO dataset, it boosts performance by 3%. |
ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laymen (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies have focused on the use of court judgments as input for legal Statute Identification (LSI) however, there is little research to explore the differences between court and laypeople data for LSI. |
| Approach: | They create a corpus of laypeople queries covering 500+ statutes from Indian law . they use court case judgements to compare between the two datasets . |
| Outcome: | The proposed corpus of laypeople queries covers 500+ statutes from Indian law . the results show that models trained on court judgements are ineffective . |
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning (N19-1)
Copied to clipboard
| Challenge: | Text adventure games provide a platform for exploring reinforcement learning in combinatorial action space, such as natural language. |
| Approach: | They propose a deep reinforcement learning architecture that represents the game state as a knowledge graph which is learned during exploration. |
| Outcome: | The proposed architecture can learn a control policy faster than baseline alternatives. |
Recursive Tree-Structured Self-Attention for Answer Sentence Selection (2021.acl-long)
Copied to clipboard
| Challenge: | Recent top-performing models in Answer Sentence Selection use self-attention and transfer learning, but not syntactic structure. |
| Approach: | They propose a recursive, tree-structured self-attention model that can represent all levels of syntactic parse trees with only one additional layer. |
| Outcome: | The proposed model can represent all levels of syntactic parse trees with only one additional layer without transfer learning. |
Testing Large Language Models on Compositionality and Inference with Phrase-Level Adjective-Noun Entailment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies have shown that pre-trained large language models acquire knowledge during pre-training which enables reasoning over relationships between words and more complex inferences over larger units of meaning. |
| Approach: | They propose a benchmark to test compositional entailment models using adjective-noun phrases. |
| Outcome: | The proposed model can generalise well to out–of–distribution sets, since the required knowledge can be stored in the representations of subwords (SW) tokens. |
Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to generalization to resource-rich languages are difficult . a recent study shows that word representations can be useful in low resource languages . |
| Approach: | They propose two approaches for improving generalization to low-resource languages by adapting continuous word representations using linguistically motivated subword units. |
| Outcome: | The proposed method improves generalization to low resource languages . it requires neither parallel corpora nor bilingual dictionaries and requires no parallel training . |
Bazinga! A Dataset for Multi-Party Dialogues Structuring (2022.lrec-1)
Copied to clipboard
Paul Lerner, Juliette Bergoënd, Camille Guinaudeau, Hervé Bredin, Benjamin Maurice, Sharleyne Lefevre, Martin Bouteiller, Aman Berhe, Léo Galmant, Ruiqing Yin, Claude Barras
| Challenge: | a dataset of 16 TV and movie series is filled with challenging multi-party dialogues. |
| Approach: | They propose a dataset built around 16 TV and movie series with challenging multi-party dialogues. |
| Outcome: | The proposed dataset is a step towards better multi-party dialogue structuring and understanding. |
GoEmotions: A Dataset of Fine-Grained Emotions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing datasets for language-based emotion classification are limited and small . existing datasets lack quality annotations for many different emotion categories . |
| Approach: | They propose to use a large manually annotated dataset to study emotion expressions . they conduct transfer learning experiments with existing emotion benchmarks to test their model . |
| Outcome: | The proposed model achieves an average F1-score of .46, leaving room for improvement. |
A Corpus for Commonsense Inference in Story Cloze Test (2022.lrec-1)
Copied to clipboard
| Challenge: | Story Cloze Test (SOTA) models can achieve over 90% accuracy on predicting the last sentence, but high accuracy can be achieved by merely using surface-level features. |
| Approach: | They constructed a human-labeled and human-verified commonsense knowledge inference dataset using data from 1871 stories and three human workers labeled each story. |
| Outcome: | The proposed models can achieve 90% accuracy on predicting the last sentence, but they don't perform well on new and more challenging tasks. |
An Interpretable Neural Network with Topical Information for Relevant Emotion Ranking (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that the extracted emotion-associated topic words represent emotion-evoking events. |
| Approach: | They propose an interpretable neural network approach for relevant emotion ranking . they initialize the hidden layer to approximate the behavior of topic models . |
| Outcome: | The proposed approach performs better than state-of-the-art methods on real-world corpora. |
WARP: Word-level Adversarial ReProgramming (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to transfer learning from pretrained language models are frozen and a task-specific head is trained on top of them. |
| Approach: | They propose an alternative approach that trains one or more task-specific layers on top of the language model. |
| Outcome: | The proposed approach outperforms existing methods on the GLUE leaderboard with just 32 training samples. |
Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition (2025.findings-acl)
Copied to clipboard
| Challenge: | Using the Wojood framework, we compare existing Arabic Named Entity Recognition models with domain and dialect divergence and resource scarcity. |
| Approach: | They propose a multi-dimensional Arabic named entity corpus covering 16 dialects across 10 domains and an annotation scheme using the Wojood guidelines. |
| Outcome: | The proposed model performs better on 16 dialects across 10 domains and 16 domains, while other models struggle with different dialects and domains. |
AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to fine-tune a language model with a large corpus in a general domain are suboptimal for downstream data when domain discrepancy exists. |
| Approach: | They propose to consider the pretrained vocabulary as an optimizable parameter . they add domain specific vocabulary based on a tokenization statistic . their method achieved consistent performance improvements on diverse domains . |
| Outcome: | The proposed method achieves consistent performance improvements on diverse domains. |
Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that transfer learning works best when the languages are related. |
| Approach: | They propose to pre-order assisting language sentences to match the word order of the source language and train the parent model. |
| Outcome: | The proposed model can improve translation quality in low-resource scenarios by pre-ordering the assisting language sentences to match the word order of the source language and training the parent model. |
How to Parse a Creole: When Martinican Creole Meets French (2022.coling-1)
Copied to clipboard
| Challenge: | a lack of annotated gold standard data is a major challenge for underresourced languages. |
| Approach: | They propose to use a French treebank to develop a dependency parser for Martinican Creole. |
| Outcome: | The proposed model is based on a French treebank and has 80 Martinican Creole sentences. |
Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics (2022.findings-emnlp)
Copied to clipboard
Hyundong Cho, Chinnadhurai Sankar, Christopher Lin, Kaushik Sadagopan, Shahin Shayandeh, Asli Celikyilmaz, Jonathan May, Ahmad Beirami
| Challenge: | Recent studies have revealed the vulnerability of dialogue state tracking models to distributional shifts, resulting in poor performance. |
| Approach: | They present a toolkit for standardized and comprehensive dialogue state tracking diagnoses that provides a richer summary of strengths and weaknesses. |
| Outcome: | The proposed toolkit shows that different classes of DST models have clear strengths and weaknesses, while generation models are more promising for handling language variety and span-based classification models are robust to unseen entities. |
Maximum Bayes Smatch Ensemble Distillation for AMR Parsing (2022.naacl-main)
Copied to clipboard
| Challenge: | AMR parsing has experienced an unprecendented increase in performance in the last three years due to a mixture of effects including architecture improvements and transfer learning. |
| Approach: | They propose to combine Smatch-based ensembling techniques with ensemble distillation to overcome this diminishing returns of silver data. |
| Outcome: | The proposed technique can produce gains rivaling those of human annotated data for QALD-9 and achieve a new state-of-the-art for BioAMR. |
Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to train NMT models rely on sparse parallel data . a variety of PC variants yield significant improvements for low-resource NMT . |
| Approach: | They propose to transfer well-trained NMT models to low-resource languages by bidirectionally-adaptive learning strategy . they divide inner constituents of Parent encoder into two "teams" aiming to adapt to characteristics of low- and high-resourced languages . |
| Outcome: | The proposed method improves on low-resource NMT models with a variety of PC variants. |
TaPas: Weakly Supervised Table Parsing via Pre-training (2020.acl-main)
Copied to clipboard
| Challenge: | Answering natural language questions over tables is often seen as a semantic parsing task. |
| Approach: | They propose an approach to question answering over tables without generating logical forms by selecting table cells and optionally applying a corresponding aggregation operator. |
| Outcome: | The proposed approach outperforms or rivals existing models on three different datasets and performs on par with the state-of-the-art on WikiSQL and WikiTQ. |
Detection and Positive Reconstruction of Cognitive Distortion Sentences: Mandarin Dataset and Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have investigated the application of NLP models in English for each stage of this process. |
| Approach: | They propose a Positive Reconstruction Framework based on broaden-and-build theory to address and reframe negative thoughts through a positive reinterpretation. |
| Outcome: | The proposed framework is based on broaden-and-build theory and can detect cognitive distortions and suggest a positive reframe in Mandarin. |
Evaluating the Values of Sources in Transfer Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Transfer learning is a form of learning that adapts a model trained on data-rich sources to low-resource targets. |
| Approach: | They propose a source valuation framework that quantifies the usefulness of the sources in transfer learning by using the Shapley value method. |
| Outcome: | The proposed framework is effective in choosing useful transfer sources and the source values match the intuitive source-target similarity. |
A Two-Stage Approach towards Generalization in Knowledge Base Question Answering (2022.findings-emnlp)
Copied to clipboard
Srinivas Ravishankar, Dung Thai, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Tahira Naseem, Pavan Kapanipathi, Gaetano Rossiello, Achille Fokoue
| Challenge: | Existing approaches for Knowledge Base Question Answering focus on a specific knowledge base or evaluating it on underlying knowledge base requires non-trivial changes. |
| Approach: | They propose a framework that separates semantic parsing from knowledge base interaction . they propose KBQA framework that allows generalization across knowledge bases . |
| Outcome: | The proposed framework achieves comparable or state-of-the-art performance on datasets with a different knowledge base. |
Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets on spatial language processing are either synthetic or at small scale. |
| Approach: | They propose a dataset for transfer learning on spatial question answering and spatial role labeling that includes a larger variety of spatial relation types and spatial expressions. |
| Outcome: | The proposed dataset can be used to evaluate spatial language processing models in real-world situations. |
Evaluation of Transfer Learning and Domain Adaptation for Analyzing German-Speaking Job Advertisements (2022.lrec-1)
Copied to clipboard
| Challenge: | a paper presents text mining approaches on German-speaking job advertisements . transfer learning and domain adaptation are used to build text mining applications . |
| Approach: | They propose text mining approaches on German-speaking job advertisements . they use transfer learning and domain adaptation to build language models adapted to job ads . |
| Outcome: | The proposed approaches outperform general-domain language models pre-trained on ten times more data. |
Cross-lingual Transfer Learning for Grammatical Error Correction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies on English GEC have focused on improving it, but the resources required to train the models are not sufficient. |
| Approach: | They investigate cross-lingual transfer learning in grammatical error correction tasks . similarities between these languages is a key factor for successfully transferring grammatikal knowledge . |
| Outcome: | The proposed methods improve accuracy of grammatical error correction tasks in English and Russian, but lack the resources to train models in these languages. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for sarcasm detection are limited due to the difficulty in acquiring ground-truth annotations. |
| Approach: | They propose a generalized latent optimization strategy that allows different losses to accommodate each other and improves training dynamics. |
| Outcome: | The proposed approach outperforms transfer learning and meta-learning baselines and achieves 10.02% performance gain on the iSarcasm dataset. |
Evaluating Pixel Language Models on Non-Standardized Languages (2025.coling-main)
Copied to clipboard
| Challenge: | pixel-based models can be used to transfer learning from standard languages to dialects . pretrained language models achieve strong results for languages seen during training, but their performance declines with out-of-domain dialects. |
| Approach: | They compare pixel-based models to token-based ones to evaluate dialects . standard german is tokenized in a more meaningful way, whereas the Bern dialect is tokenize in pixel form . |
| Outcome: | The proposed models outperform token-based models in part-of-speech tagging, dependency parsing and intent detection for zero-shot dialect evaluation by up to 26 percentage points in some scenarios, though not in Standard German. |
Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show pre-trained language models contain matching subnetworks that have similar transfer learning performance as the original PLM. |
| Approach: | They propose to prune matching subnetworks using magnitude-based pruning . they propose to optimize the subnetwork structure towards the pre-training objectives . |
| Outcome: | The proposed method is more efficient in searching subnetworks and advantageous when fine-tuning within a range of data scarcity. |
Being Generous with Sub-Words towards Small NMT Children (2020.lrec-1)
Copied to clipboard
| Challenge: | In the context of under-resourced neural machine translation, transfer learning from an NMT model trained on a high resource language pair, or from a multilingual NMT (M-NMT) model, has been shown to boost performance to a large extent. |
| Approach: | They propose to use a multilingual NMT model to train on an under-resourced child and to use large sub-word vocabularies to improve performance. |
| Outcome: | The proposed approach involving dynamic vocabularies is both practical and effective on two under-resourced language pairs, i.e. Icelandic-English and Irish-English. |
Transfer Learning Methods for Domain Adaptation in Technical Logbook Datasets (2022.lrec-1)
Copied to clipboard
| Challenge: | Technical logbook data typically has both a domain, the field it comes from, and an application, what it is used for. |
| Approach: | They propose to use domain-specific technical language to identify technical logbook entries by using transfer learning to learn from different domains and from different datasets. |
| Outcome: | The proposed approach improves performance in all cases but one of the three domains studied. |
JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) requires large parallel corpora for training robust and high quality models. |
| Approach: | They propose a Japanese-specific sequence to sequence pre-training alternative to MASS for NMT . they use Japanese as the source or target language to train their models . |
| Outcome: | The proposed approach can give competitive results over MASS and BRSS, and significantly surpass the individual methods. |
Culture Matters in Toxic Language Detection in Persian (2025.acl-long)
Copied to clipboard
| Challenge: | Toxic language detection is crucial for creating safer online environments and limiting the spread of harmful content. |
| Approach: | They compare different methods for toxic language detection in Persian to fine-tune, enrich data, and cross-lingual transfer learning. |
| Outcome: | The language of a country with cultural similarities to Persian yields better results in transfer learning. |
Could you give me a hint ? Generating inference graphs for defeasible reasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | Defeasible reasoning is a mode of reasoning where conclusions can be overturned by taking into account new evidence. |
| Approach: | They propose to automatically generate inference graphs for a defeasible inference task by transfer learning from a related NLP task. |
| Outcome: | The proposed method generates meaningful graphs for a defeasible inference task and human accuracy improves by 20%. |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing web crawling pipelines are used to collect large corpora raw data, but the main way to collect such data is through manual data extraction. |
| Approach: | They propose to use a web crawler to extract and classify data from a multilingual web corpus and an automated annotation pipeline to improve it. |
| Outcome: | The proposed version of OSCAR could be used to pre-train large generative language models and other applications in Natural Language Processing and Digital Humanities. |
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020.acl-main)
Copied to clipboard
| Challenge: | a new evaluation paradigm, Pretraining-Agnostic Identically Distributed evaluation, is needed . authors argue that it rewards models that can be trained on massive amounts of data, several orders of magnitude more than a human can expect to be exposed to. |
| Approach: | a position paper describes and critiques the Pretraining-Agnostic Identically Distributed evaluation paradigm . paradigm favors simple, low-bias architectures that can be scaled to process vast amounts of data . authors advocate for supplementing or replacing PAID with paradigms that reward architectures . |
| Outcome: | a new evaluation paradigm favors simple, low-bias architectures that can be scaled to process vast amounts of data. a san francisco-based study finds that the paradigm rewards architectures which generalize as quickly and robustly as humans. |
Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkien (2022.findings-emnlp)
Copied to clipboard
| Challenge: | CM is a challenging task when mixed languages include dialects. |
| Approach: | They propose to construct a Hokkien-Mandarin CM dataset to overcome the limitation . they propose to use a linguistics-based toolkit to train the model for translation tasks . |
| Outcome: | The proposed model achieves good results on CM data translation while maintaining monolingual translation quality. |
Multilingual Offensive Language Identification with Cross-lingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Several studies investigating methods to detect offensive content in social media use English data. |
| Approach: | They apply cross-lingual contextual embeddings and transfer learning to make predictions in languages with less resources. |
| Outcome: | The proposed method compares favorably to the best systems submitted to recent shared tasks on Bengali, Hindi, and Spanish. |
Multilingual Models for ASR in Chibchan Languages (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing algorithms for low resource-intensive languages are not available for these languages . a paper comparing the performance of different models and algorithms for these extremely low resource languages is presented. |
| Approach: | They propose to fine-tune four ASR algorithms to create monolingual models for Bribri and Cabécar . they then use the best performing algorithm to train joint and transfer learning models for both languages . |
| Outcome: | The proposed algorithms are effective in both Bribri and Cabécar, but especially in Bribri. |
MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for dialogue summarization are limited to their small sizes and are built from a narrow domain. |
| Approach: | They propose a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. |
| Outcome: | The proposed dataset is larger and contains multi-party conversations from multiple domains. |
ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots (2025.naacl-long)
Copied to clipboard
Yu-Chung Hsiao, Fedir Zubach, Gilles Baechler, Srinivas Sunkara, Victor Carbune, Jason Lin, Maria Wang, Yun Zhu, Jindong Chen
| Challenge: | Existing screen datasets focus on low-level structural and component understanding or on a much higher-level composite task such as navigation and task completion for autonomous agents. |
| Approach: | They propose to annotate 86k question-answer pairs over the RICO dataset to benchmark screen content understanding. |
| Outcome: | The proposed dataset covers full answers, short answer phrases, and corresponding UI contents with bounding boxes, enabling four subtasks to address various application scenarios. |
A New Dataset for Topic-Based Paragraph Classification in Genocide-Related Court Transcripts (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have lowered the barriers for people outside the NLP community to tap into the tools and resources applied to a variety of domain-specific applications. |
| Approach: | They propose to annotate court transcripts from genocide-related cases using transformer-based approaches and to establish benchmarks for the task of paragraph identification of violence-related witness statements. |
| Outcome: | The first annotated corpus of genocide-related court transcripts is aimed at providing a first reference corpus for the community and to establish benchmark performances using state-of-the-art transformer-based approaches. |
Leveraging Medical Literature for Section Prediction in Electronic Health Records (D19-1)
Copied to clipboard
| Challenge: | Prior approaches to section prediction have only used text data from EHRs and required significant manual annotation. |
| Approach: | They propose to use sections from medical literature to train models to predict sections in EHRs. |
| Outcome: | The proposed model uses sections from medical literature that contain similar content to those found in EHR sections. |
Emancipating Event Extraction from the Constraints of Long-Tailed Distribution Data Utilizing Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for EE depend on manual annotations, which are expensive and scarce. |
| Approach: | They propose to transform the event extraction task into multi-turn dialogues and a novel method for generating high-quality data. |
| Outcome: | The proposed methods significantly improve existing models’ performance with various paradigms and structures, especially on tail types. |
PRADO: Projection Attention Networks for Document Classification On-Device (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of on-device neural networks for long text classification. |
| Approach: | They propose a projection attention neural network PRADO that combines trainable projections with attention and convolutions to train tiny neural networks that achieve high performance on multiple long document classification tasks. |
| Outcome: | The proposed model achieves high performance on multiple long document classification tasks while maintaining compact size. |
Learning Dense Representations of Phrases at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Existing phrase retrieval models rely on sparse representations and still underperform retriever-reader approaches. |
| Approach: | They propose a method to learn phrase representations from reading comprehension tasks using negative sampling methods. |
| Outcome: | The proposed model improves over previous models by 15%-25% absolute accuracy and matches the performance of state-of-the-art retrieval models. |
Common Voice: A Massively-Multilingual Speech Corpus (2020.lrec-1)
Copied to clipboard
Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, Gregor Weber
| Challenge: | Common Voice is a massively-multilingual collection of transcribed speech intended for speech technology research and development. |
| Approach: | They propose to use Mozilla’s DeepSpeech Speech-to-Text toolkit to perform multilingual automatic speech recognition experiments. |
| Outcome: | The proposed corpus is the largest in the public domain for speech recognition, both in terms of hours and languages. |
A Case Against Implicit Standards: Homophone Normalization in Machine Translation for Languages that use the Ge’ez Script. (2025.emnlp-main)
Copied to clipboard
Hellina Hailu Nigatu, Atnafu Lambebo Tonja, Henok Biadglign Ademtew, Hizkiel Mitiku Alemayehu, Negasi Haile Abadi, Tadesse Destaw Belay, Seid Muhie Yimam
| Challenge: | Homophone normalization is a pre-processing step used in Amharic natural language processing (NLP) but it also results in models that are unable to process different forms of writing in a single language. |
| Approach: | They propose a method where normalization is applied to model predictions instead of training data and a scheme where normalized data is preserved in training. |
| Outcome: | The proposed model achieves an increase in BLEU score of up to 1.03 while preserving language features in training. |
Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prior methods producing useful task rankings are infeasible for large source pools . Embedding space maps (ESMs) reduce execution time and disk space usage . |
| Approach: | They introduce Embedded Space Maps (ESMs) that approximate the effect of fine-tuning a language model. |
| Outcome: | The proposed method reduces execution time and disk space usage by 10 and 278, respectively, while retaining high selection performance. |
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual models cannot fully leverage training data when it is available in different task-language combinations. |
| Approach: | They propose a single hypernetwork that unifies multi-task and multilingual learning with efficient adaptation. |
| Outcome: | The proposed model achieves the best or competitive gain when a mixture of multiple resources is available while being significantly more efficient than existing models. |
A Survey of Multilingual Models for Automatic Speech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, but the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models. |
| Approach: | They propose to use unlabeled speech data to build multilingual ASR models that can be used for improved performance on low-resource languages. |
| Outcome: | The proposed models can be used to improve performance on low-resource languages by using unlabeled speech data. |
Transfer Fine-Tuning: A BERT Case Study (D19-1)
Copied to clipboard
| Challenge: | Recent advances in sentence representation learning have improved the performance of BERT models, but the computational power required is an obstacle preventing practical applications from adopting the technology. |
| Approach: | They propose to inject phrasal paraphrase relations into BERT to generate suitable representations for semantic equivalence assessment instead of increasing model size. |
| Outcome: | The proposed model improves a smaller model while maintaining the model size. |
Learning to Predict Task Transferability via Soft Prompt (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that fine-tuning pretrained language models on helpful intermediate tasks yields further gains. |
| Approach: | They propose to train an affinity scoring function to predict transferability between tasks by conditioning on task embeddings. |
| Outcome: | The proposed method efficiently identifies beneficial tasks for transfer learning. |
Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2018a, 2018b) show that LSTMs can transfer from non-linguistic data to natural language models with different types of abstract structure. |
| Approach: | They propose to use transfer learning to analyze encoding of grammatical structure in neural language models. |
| Outcome: | The proposed method improves test performance on natural language despite no overlap in surface form or vocabulary. |
Evaluating Unsupervised Representation Learning for Detecting Stances of Fake News (2020.coling-main)
Copied to clipboard
| Challenge: | Using unsupervised representation learning, automated Fake News detection is a challenge for researchers. |
| Approach: | They examine pre-trained language models with respect to their performance on two Fake News related data sets. |
| Outcome: | The proposed models outperform the autoregression-based models on two Fake News related data sets. |
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP. |
| Approach: | They propose two methods to train multilingual graphs from colexification patterns using an unannotated parallel corpus. |
| Outcome: | The proposed methods achieve high recall on CLICS and transfer learning in multilingual graphs. |
RoBERT – A Romanian BERT Model (2020.coling-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models learn contextualized representations by using unlabeled text data and obtain state of the art results on a multitude of NLP tasks. |
| Approach: | They propose a pre-trained BERT model for Romanian language processing and compare it with multi-lingual models on seven Romanian specific NLP tasks. |
| Outcome: | The proposed model outperforms multi-lingual models on seven Romanian specific NLP tasks on sentiment analysis, dialect and cross-dialect topic identification, and diacritics restoration. |
Low-resource Deep Entity Resolution with Transfer and Active Learning (P19-1)
Copied to clipboard
| Challenge: | Entity resolution (ER) is the task of identifying different representations of the same real-world entities across databases. |
| Approach: | They propose a deep learning-based method that targets low-resource settings for ER by combining transfer learning and active learning. |
| Outcome: | The proposed method achieves comparable, if not better, performance compared to state-of-the-art learning-based methods while using an order of magnitude fewer labels. |
Out-of-Task Training for Dialog State Tracking Models (2020.coling-main)
Copied to clipboard
Michael Heck, Christian Geishauser, Hsien-chin Lin, Nurul Lubis, Marco Moresi, Carel van Niekerk, Milica Gasic
| Challenge: | Dialog state tracking (DST) suffers from data sparsity. |
| Approach: | They utilize non-dialog data from unrelated NLP tasks to train dialog state trackers . they propose to use dialog state tracking to summarise the conversation history . |
| Outcome: | The proposed method exploits non-dialog data from unrelated NLP tasks to train dialog state trackers. |
“I’d rather just go to bed”: Understanding Indirect Answers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Humans produce and interpret complex utterances even in simple scenarios. |
| Approach: | They present a large-scale English language corpus with 34,268 (polar question, indirect answer) pairs to enable progress on this task. |
| Outcome: | The proposed corpus contains 34,268 (polar question, indirect answer) pairs, and reaches 82-88% accuracy for a 4-class distinction, and 64-85% for 6 classes. |
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels (2020.emnlp-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos
| Challenge: | Large-scale Multi-label Text Classification (LMTC) is a type of classification that assigns labels to a large set of labels. |
| Approach: | They propose to use probabilistic label trees to improve frequent, few and zero-shot learning . they propose to combine a new state-of-the-art method with pre-trained Transformers . |
| Outcome: | The proposed models outperform existing models on frequent, few and zero-shot learning on three datasets from different domains. |
RADS: Reinforcement Learning-Based Sample Selection Improves Transfer Learning in Low-resource and Imbalanced Clinical Settings (2026.findings-acl)
Copied to clipboard
| Challenge: | Experimental evaluations show that RL methods favor outliers rather than truly informative samples under low-resource and class-imbalanced conditions. |
| Approach: | They propose a robust sample selection strategy using reinforcement learning to identify the most informative samples using a class imbalance approach. |
| Outcome: | The proposed strategy improves model transferability while maintaining robust performance under extreme class imbalance compared to traditional methods. |
PatchBERT: Just-in-Time, Out-of-Vocabulary Patching (2020.emnlp-main)
Copied to clipboard
| Challenge: | a pre-trained language model with low OOV can improve performance for transfer learning . a vocabulary surrogate can provide performance boosts with no additional computation cost . |
| Approach: | They propose multiple methods to mitigate OOV during downstream task fine-tuning . they demonstrate that vocabulary surrogates can provide performance boosts with no additional computation cost . |
| Outcome: | The proposed methods improve performance with the same parameter count when combined with fine-tuning. |
KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge-driven dialog data is limited due to the lack of dialog data which consists of multi-turn conversations on multiple topics and with knowledge annotations. |
| Approach: | They propose a Chinese multi-domain knowledge-driven conversation dataset which grounds the topics in multi-turn conversations to knowledge graphs. |
| Outcome: | The proposed dataset can be enhanced by introducing background knowledge, but there is still a large space for leveraging knowledge to model multi-turn conversations for further research. |
Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)
Copied to clipboard
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
| Challenge: | Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. |
| Approach: | They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems. |
| Outcome: | The proposed model can improve performance even with low-data source tasks that differ substantially from the target task. |
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning (2023.acl-long)
Copied to clipboard
| Challenge: | Experimental results show zero-shot performance on unseen multimodal tasks . instruction tuning has yet to be explored for vision and multimodal task. |
| Approach: | They propose a multimodal instruction tuning benchmark dataset that consists of 62 diverse multimodal tasks in a unified seq-to-seq format covering 10 broad categories. |
| Outcome: | The proposed model performs well on unseen multimodal tasks and is highly scalable. |
DoQA - Accessing Domain-Specific FAQs via Conversational QA (2020.acl-main)
Copied to clipboard
| Challenge: | a dataset of 2,437 dialogues and 10,917 QA pairs is used to access domain-specific FAQ information. |
| Approach: | They present a dataset with 2,437 dialogues and 10,917 QA pairs for FAQs . they use the Wizard of Oz method with crowdsourcing to create dialogues using the original post and the original reply. |
| Outcome: | The proposed system can access domain-specific FAQ information without training data. |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Challenging the Assumption of Structure-based embeddings in Few- and Zero-shot Knowledge Graph Completion (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing work on Knowledge Graph completion only uses textual descriptive data . knowledge graphs are incomplete because not every relation has been observed at the time of their construction. |
| Approach: | They propose to use textual descriptive data to enrich benchmark data sets for Few- and Zero-shot Knowledge Graph completion tasks. |
| Outcome: | The proposed task improves for Few- and Zero-shot scenarios with up to twofold increase in the Zero- shot setting. |
In Neural Machine Translation, What Does Transfer Learning Transfer? (2020.acl-main)
Copied to clipboard
| Challenge: | a recent study found that word embeddings are not necessary for transfer learning. |
| Approach: | They perform several ablation studies that limit information transfer and measure the quality impact across three language pairs to gain a black-box understanding of transfer learning. |
| Outcome: | The proposed method can eliminate the need for a warm-up phase when training transformer models in high resource language pairs. |
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem (2020.acl-main)
Copied to clipboard
| Challenge: | Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. |
| Approach: | They propose a lattice-rescoring scheme which allows a trade-off between general translation quality and bias reduction during adaptation and inference time. |
| Outcome: | The proposed approach outperforms all systems evaluated on WinoMT with no degradation of general test set BLEU. |
KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for data-to-text generation rely on labeled data, which is costly to acquire and limits their application to new tasks and domains. |
| Approach: | They propose to leverage pre-training and transfer learning to address this problem by leveraging a general knowledge-grounded generation model and a knowledge-based model. |
| Outcome: | The proposed model can generate knowledge-enriched text on a knowledge-grounded text corpus crawled from the web in three settings. |
PRO-CS : An Instance-Based Prompt Composition Technique for Code-Switched Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Code-switching (CS) data is ubiquitous in today’s globalized world, but the dearth of annotated datasets in code-switch tasks poses a significant challenge for transfer learning in limited-resource setups. |
| Approach: | They propose a prompt composition technique that outperforms prompt-tuning and fine-tuned prompt-based prompt composition techniques for CS tasks that combine language and task knowledge. |
| Outcome: | The proposed approach outperforms prompt-tuning and fine-tuned approaches on 10 datasets across 4 languages and achieves competitive results in low-resource cross-lingual and cross-task setting. |
ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale (2024.findings-acl)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) has shown significant practical benefits when using language models . current two stage MTL introduces a substantial number of additional parameters . |
| Approach: | They propose a multi-task learning method that leverages existing knowledge for a target task. |
| Outcome: | The proposed method outperforms baselines on three benchmarks and two encoder LMs with a small number of transfer parameters. |
Neural Machine Translation Methods for Translating Text to Sign Language Glosses (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language glosses. |
| Approach: | They propose to use data augmentation, semi-supervised Neural Machine Translation, transfer learning and multilingual NMT to improve MT of spoken language to Sign Language glosses. |
| Outcome: | The proposed models outperform previous work on two German SL corpora and are confirmed by human evaluation. |
GreekBART: The First Pretrained Greek Sequence-to-Sequence Model (2024.lrec-main)
Copied to clipboard
Iakovos Evdaimon, Hadi Abdine, Christos Xypolopoulos, Stamatis Outsios, Michalis Vazirgiannis, Giorgos Stamou
| Challenge: | Transfer learning has revolutionized the fields of Computer Vision and Natural Language Processing. |
| Approach: | They introduce a new language model, GreekBART, that is based on a BART-base architecture. |
| Outcome: | The proposed model outperforms BERT, GPT and other transformer-based models on discriminative tasks. |
ClusterPrompt: Cluster Semantic Enhanced Prompt Learning for New Intent Discovery (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for identifying new intent categories focus on relations between utterances and clusters, while neglecting the usage of semantics. |
| Approach: | They propose a method that leverages contrastive learning and label semantic alignment to learn meaningful representations of intent clusters. |
| Outcome: | The proposed method outperforms existing methods and suggests meaningful intent labels. |
Transfer Learning for Named-Entity Recognition with Neural Networks (L18-1)
Copied to clipboard
| Challenge: | Existing approaches to named-entity recognition (NER) require additional lead time for developing and fine-tuning the rules. |
| Approach: | They propose to transfer an ANN model trained on a large labeled dataset to another dataset with a limited number of labels to improve upon the state-of-the-art results for patient note de-identification. |
| Outcome: | The proposed model can be transferred to a dataset with a limited number of labels, and improves on the state-of-the-art results on patient note de-identification. |
An Empirical Study on the Transferability of Transformer Modules in Parameter-efficient Fine-tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning is a computationally expensive process . introducing new parameters to an already-large model can be considered a drawback. |
| Approach: | They investigate the capability of different transformer modules in transferring knowledge from a pre-trained model to a downstream task. |
| Outcome: | The proposed methods show that each transformer module is a winning ticket . they show that with only 0.003% updateable parameters, they can show acceptable performance on target tasks. |
A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Lip reading is a visual observation of a speaker's lips that can be used for communication problems. |
| Approach: | They present a dataset of 250,000 publicly available videos of speakers of the Hessian Parliament which was processed for word-level lip reading using an automatic pipeline. |
| Outcome: | The proposed dataset GLips (German Lips) is compared with the LRW dataset and shows that it has language-independent features. |
MedDialog: Large-scale Medical Dialogue Datasets (2020.emnlp-main)
Copied to clipboard
Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie
| Challenge: | telemedicine is a medical practice that provides patient care remotely using video conferencing tools. |
| Approach: | They build large-scale medical dialogue datasets to facilitate research . they pretrain several models on the Chinese MedDialog dataset and compare their performance . |
| Outcome: | The proposed datasets show that models trained on MedDialog can generate doctor-like medical dialogues. |
Pars-ABSA: a Manually Annotated Aspect-based Sentiment Analysis Benchmark on Farsi Product Reviews (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing systems for sentiment analysis are focused on document and sentence levels, but there are no public datasets on aspect-based sentiment analysis for Farsi. |
| Approach: | They propose to use a manually annotated Farsi dataset to analyze the opinion polarity of reviews . they also use transfer learning to analyze aspects of the review to improve their results . |
| Outcome: | The proposed method performs better than other aspects of the existing system. |
Wasserstein Selective Transfer Learning for Cross-domain Text Mining (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve the learning of data-scarce target domains have negative transfer due to the data distributions between source and target domain. |
| Approach: | They propose a method that uses a reinforced selector to select helpful data for transfer learning and a Wasserstein-based discriminator to maximize the distance between the selected data and target data. |
| Outcome: | The proposed method performs better on three real-world text mining tasks. |
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture (2023.findings-emnlp)
Copied to clipboard
Bingsheng Yao, Ishan Jindal, Lucian Popa, Yannis Katsis, Sayan Ghosh, Lihong He, Yuxuan Lu, Shashank Srivastava, Yunyao Li, James Hendler, Dakuo Wang
| Challenge: | Existing low-resource learning techniques focus on label annotation while neglecting the natural language explanation of a data point. |
| Approach: | They propose a novel architecture that leverages an explanation-generation model to produce explanations guided by human explanations and a prediction model that utilizes generated explanations toward prediction faithfully. |
| Outcome: | The proposed architecture produces explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a data diversity-based AL sampling strategy that benefits from the explanation annotations. |
RoBERTuito: a pre-trained language model for social media text in Spanish (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained language models have been used in many natural language processing tasks . some domain-specific models have shown to improve performance in some domains . however, for languages other than English, such models are not widely available . |
| Approach: | They present a pre-trained language model for user-generated text in Spanish . it is based on 500 million tweets and has some cross-lingual abilities . |
| Outcome: | The model outperforms models trained on over 500 million tweets on a benchmark in spanish and english. |
Multilingual Transfer Learning for Children Automatic Speech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent advances in automatic speech recognition (ASR) systems have been criticized for high acoustic variability and limited amount of available training data. |
| Approach: | They propose a two-step training strategy that uses multilingual learning followed by language-specific transfer learning to generalize children's speech. |
| Outcome: | The proposed training strategy outperforms single language training and multilingual and transfer learning alone in English. |
Improving Sentence Boundary Detection for Spoken Language Transcripts (2020.lrec-1)
Copied to clipboard
| Challenge: | Using data expansion and transfer learning, we find that data expansion does not always improve results. |
| Approach: | They propose to divide spoken language into sentence-like units using Topological Fields model . they also propose to use data from the same domain to test different ML architectures . |
| Outcome: | The proposed model improves the detection of boundary detection in spoken dialogues compared to a sequence tagging approach. |
“A Little is Enough”: Few-Shot Quality Estimation based Corpus Filtering improves Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Quality Estimation (QE) is the task of evaluating the quality of a translation when reference translation is unavailable. |
| Approach: | They propose a Quality Estimation based Filtering approach to extract high-quality parallel data from the pseudo-parallel corpus. |
| Outcome: | The proposed approach improves the machine translation system performance by up to 1.8 BLEU points over the baseline model. |
Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning? (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are highly effective tools for solving different kinds of problems in natural language processing. |
| Approach: | They propose to use large language models to solve a myriad of problems. |
| Outcome: | The proposed model performs worse on word meaning comprehension than an encoder-only model with vastly fewer parameters. |
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that multi-head attention module evolves functional specialization after multi-task training. |
| Approach: | They propose a method to quantify the degree of functional specialization in multi-head attention . they propose 'multi-task training' method to increase functional specialisation and mitigate negative information transfer . |
| Outcome: | The proposed method increases functional specialization and mitigates negative information transfer in multi-task learning without adding any parameters. |
Searching for the Most Human-like Emergent Language (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on emergent communication systems to generate languages with high statistical similarity to human languages has not been done. |
| Approach: | They propose to optimize a signalling game-based emergent communication environment to generate state-of-the-art emergentic languages with a high degree of similarity to human language. |
| Outcome: | The proposed language generates state-of-the-art on XferBench benchmark, demonstrating its similarity to human language and entropy-minimization properties. |
Logos as a Well-Tempered Pre-train for Sign Language Recognition (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing data on sign languages is limited, resulting in ambiguity in dataset labeling . similar signs can have different semantic meanings, which leads to ambiguous labeling. |
| Approach: | They propose to use a Russian sign language dataset as a universal encoder for other languages . they propose to explicitly annotate visually similar signs to improve model quality . |
| Outcome: | The proposed model outperforms current state-of-the-art models and gets competitive results for the AUTSL dataset. |
Strengthening the WiC: New Polysemy Dataset in Hindi and Lack of Cross Lingual Transfer (2024.lrec-main)
Copied to clipboard
| Challenge: | a new study addresses the problem of natural language processing in low-resource languages such as Hindi . the paper focuses on Word Sense Disambiguation, a fundamental NLP task that deals with polysemous words. |
| Approach: | They propose a Hindi WSD dataset that allows training and testing of contextualized models. |
| Outcome: | The proposed dataset enables training and testing of contextualized models in Hindi . the results show that the proposed dataset can handle polysemy tasks in low-resource languages . |
TAPASGO: Transfer Learning towards a German-Language Tabular Question Answering Model (2024.lrec-main)
Copied to clipboard
| Challenge: | This paper examines the performance and limitations of fine-tuned models for tabular data analysis. |
| Approach: | They propose to use fine-tuning mechanics to fine- tune an English model towards a potential German model for tabular data analysis. |
| Outcome: | The proposed model outperforms the original model on German training data. |
SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sign language processing have relied on task-specific models, limiting the potential for transfer learning across tasks. |
| Approach: | They propose a self-supervised contextual representation model that adapts masked token prediction objectives to multi-stream visual sign language input. |
| Outcome: | The proposed model adapts masked token prediction objectives to multi-stream visual sign language input, learning to predict multiple targets corresponding to clustered hand, face, and body pose streams. |
TunArTTS: Tunisian Arabic Text-To-Speech Corpus (2024.lrec-main)
Copied to clipboard
| Challenge: | Historically, TTS relied on classical methods that proved expensive in terms of data storage and often resulted in robotic-sounding output known as concatenative speech. |
| Approach: | They propose to extract a mono-speaker speech corpus from an online dictionary and use it to develop end-to-end TTS systems for the Tunisian dialect. |
| Outcome: | The proposed system is based on two approaches: training from scratch and transfer learning. |